Thursday, May 21, 2020

Saturday, October 06, 2018

Python Counter

# collections.Counter lets you find the most common # elements in an iterable: >>> import collections >>> c = collections.Counter('helloworld') >>> c Counter({'l': 3, 'o': 2, 'e': 1, 'd': 1, 'h': 1, 'r': 1, 'w': 1}) >>> c.most_common(3) [('l', 3), ('o', 2), ('e', 1)] === https://realpython.com/

Saturday, September 29, 2018

Python 3.7 debugging

Python 3.7 has internal function breakpoint(). It goes into pdb if it hits that function. https://www.python.org/dev/peps/pep-0553/

I just forgot to transpose

julia> x*x ERROR: MethodError: no method matching *(::Array{Int64,1}, ::Array{Int64,1}) Closest candidates are: *(::Any, ::Any, ::Any, ::Any...) at operators.jl:502 *(::LinearAlgebra.Adjoint{#s549,#s548} where #s548<:Union{DenseArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2}, ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray}, SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray}where N where T, DenseArray}} where #s549, ::Union{DenseArray{S,1}, ReinterpretArray{S,1,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray}, ReshapedArray{S,1,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray}, SubArray{S,1,A,I,L} where L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where A<:DenseArray where N where T, DenseArray} where N where T, DenseArray}}) where {T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64}, S} at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\LinearAlgebra\src\matmul.jl:97 *(::LinearAlgebra.Adjoint{#s549,#s548} where #s548<:LinearAlgebra.AbstractTriangular where #s549, ::AbstractArray{T,1} where T) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\LinearAlgebra\src\triangular.jl:1805 ...Stacktrace: [1] top-level scope at none:0 julia> x*x'3×3 Array{Int64,2}: 1 2 3 2 4 6 3 6 9

Sunday, April 15, 2018

Purdue Online Writing Lab https://owl.english.purdue.edu/owl/

Thursday, February 09, 2017

Metadata kept in Windows cache, refused to mount

I think this is related to hibernation. I have found the solution sudo ntfsfix /dev/sda(#)

Thursday, February 02, 2017

Symmetric Dirichlet in d dimensions

function res=dirchrnd(alpha,d)
res = zeros(d,1);
for i=1:d
res(i)=gamrnd(alpha,1);
end
res=res/sum(res);
end

Thursday, September 15, 2016

Juno Atom Julia v0.5

I received this error after updating Atom packages of Julia
"unknown package Juno required by Atom"

I attempt to solve using
Pkg.clone("https://github.com/JunoLab/Juno.jl.git")
Pkg.update()
using Atom

However, in next step Gallium gave an error.
I removed package Gallium and added again.

Then with existing installation of Atom, Julia works after precompilation finishes.

Tuesday, August 30, 2016

Using Global Namespace functions

If you would like to use the global function while defining function with same name use this way :

Vector Vector::exp()
{
Vector& r = buffer.get();
for (int i = 0; i r.data[i] = ::exp(data[i]);
return buffer.next();
}

Friday, December 18, 2015

Twitter Rate Limits

Twitter allows to capture 3200 tweets in the timeline.
A java program bypasses this limit using search system
https://github.com/Jefferson-Henrique/GetOldTweets

Tuesday, November 03, 2015

Selecting Specific Elements in the matrix

Sometimes I would like to select elements in the Matlab matrix with their coordinate. However, if I used two arrays to get elements in Matlab it returns the rows and columns containing those elements. To get exact elements, I used following method where indx and indy is index coordinates:

A(sub2ind(size(A),indx,indy))

Tuesday, September 01, 2015

Named rows and columns

R has built-in support for naming rows and columns for matrices. Matlab has this functionality via dataset class

http://www.mathworks.com/matlabcentral/answers/83204-add-row-column-names-to-a-matrix

One can use double(X) to convert dataset back to matrix.

Wednesday, February 19, 2014

Creating thread with their id's

for(i=0;i < numthereads;i++)
workers.push_back(thread(&ThreadPool::thread_loop,this,i));

Thursday, January 23, 2014

Friday, January 17, 2014

Two small console debug tools

#define step() toc=tic;tic=GetTickCount();printf("Elapsed: %d\n",tic-toc);
#define pause() system("pause");

Monday, November 04, 2013

Matlab Single Value Default Function

function default(varargin)
   param = [varargin{:} ';'];
   ind = find(param=='=');
   stmt = 'if ~exist(''%s'',''var'');%s;end';
   str = sprintf(stmt,param(1:(ind(1)-1)),[varargin{:}]);
   evalin('caller', str);
end


Example use

default x=2

Tuesday, October 22, 2013

Wednesday, August 28, 2013

Good observation

"Just as machine learning algorithms can overfit by tuning a classifier’s parameters to the
accidental properties of a training set, a research community can overfit by refining algorithms that
have already done well on the existing data sets"



Lewis, D. D.; Yang, Y.; Rose, T.; and Li, F. RCV1: A New Benchmark Collection for Text Categorization Research. Journal of Machine Learning Research, 5:361-397, 2004. http://www.jmlr.org/papers/volume5/lewis04a/lewis04a.pdf

Monday, July 22, 2013

Matlab Keyboard function

keyboard function is useful for debugging purposes. It stops execution of m file and allows user to watch and manipulate variables.

Wednesday, July 10, 2013

Matlab Jit Acceleration


Monday, July 01, 2013

Moving file to folder in HG versioning system

Just copy files into desired directory. Go into that directory and right click. Go to hg->guess renames. Rest is straightforward.

http://kiln.stackexchange.com/questions/1943/moving-a-file-in-a-repository

Semi Turing Test

A loosely definition for semi-Turing test is whether a human is capable to understand an output from human and output from computer aided human. For example If I am using spell checker , how someone can understand that I am doing so. It requires much more effort to distinguish.

Wednesday, April 03, 2013

Matlab Long Simulation Interactive Debug , Set Debug flag using GUI , Pause execution to debug

Create a button function:


function util_DebugGuiButtons()
global debugFlag;
uicontrol('Position',[30 10 35 40],'String','||','Callback',@wait);
uicontrol('Position',[70 10 35 40],'String','>>','Callback',@dontwait);
function wait(hObject,eventdata)
debugFlag = 1;
end

function dontwait(hObject,eventdata)
debugFlag = 0;
dbcont;
end
end



Define in top of document:
global debugFlag


Put this inside your loop :


    if (debugFlag)
        dbstop in file.m at line
end



Continue Button is not working as desired now. It just clears the breakpoints but does not issue dbcont command properly. I will try to figure out later.

Sunday, March 17, 2013

Comments on variables

Matlab displays comments of functions when you tend to use it. It would be good if it can show the comment of variables while holding mouse on top of variable name. This way can provide a tool to remember what each variable if for without using long descriptive variable names.

Monday, March 11, 2013

Wednesday, March 06, 2013

mouse and pencil

While being a student my life is accomodated with two items   mouse and pencil.So why we are not combining them . Put a wireless mouse pen to one side and put a pencil to other side. A gyro sensor can figure out if mouse is active.

Tuesday, March 05, 2013

Robot Dish Washer

It is difficult to create robot that cleans our arbitrary dirty dishes. However  , if we put some digital information on dishes (special designed for this machine) such as RFID , than robot can easily understand the shape , coordinates and features of the dish. In this way it will be easy to optimize the placement of the dishes based on current dirty dishes information.
I imagine we can create machine that has two part , first part is regular dish washer and second part is open area for dirty dishes and attached robot hands at top. It will increase the size of the machine , but it may worth , especially for big restaurants.

Friday, February 15, 2013

Monday, January 21, 2013

Wednesday, January 02, 2013

Display Array in NetBeans

Netbeans uses GDB:

http://stackoverflow.com/questions/1651682/how-to-watch-a-c-dynamic-array-using-gdb

Friday, December 21, 2012

Lazy Functions

I imagine compiler supported lazy functions that memorizes last value of return , then never computes it again until an input variable changes , maybe keeps states of input vars. This may help to reduce computation , when reads are more frequent than writes .  A flag could be set if there is any write to specified vars

Saturday, November 24, 2012

R normalize data into 0-1 Interval


function normalize(data)
{
  maxvalues = apply(data,2,max)
  minvalues = apply(data,2,min)
  sweep(data,2,minvalues,"-")
  sweep(data,2,maxvalues-minvalues,"/")
}

Monday, November 05, 2012

R hold plot

There is no hold function R instead lines or points are used for that purpose :


plot(method1,type="l", main="Accuracy vs Number of Training samples",
      xlab="Number of samples", ylab="Accuracy",
      xlim=c(0, 30), ylim=c(0, 1))
lines(method2,col="blue")

Friday, November 02, 2012

Tanenbaum's Phrase

I liked the phrase "Researcher-push than demand-pull" in Tanenbaum's OS book.

Thursday, November 01, 2012

Wednesday, October 24, 2012

Possible Extensions for State of the Art Machine Learning Methods

-Relax the assumptions
-Put in a hierarchy
-Semi,hybrid version
-Active,online version
-Better performance
-Relation with other methods
-More efficient
-A more general version
-Application on more realistic problem or different domain

Monday, October 22, 2012

Evalution metrics using Javascript

function getEvaluationMetrics(tp,fp,tn,fn){ 
precision=(tp/(tp+fp)); 
recall=(tp/(tp+fn)); 
f1=2*(precision*recall/(precision+recall)); 
var m= new Array(3); m[0]=precision; m[1]=recall; m[2]=f1; 
return m; 
}

Monday, October 15, 2012

Paradox

nothing is exactly true -> Altough we observe many things that is not perfect , this hypothesis could not be true for any hypothesis set.

Thursday, October 11, 2012

R/Matlab Anonymous functions

R for variable X:
means <- function(x)  rnorm(1,x)

Matlab
means = @(x) x+randn(1)

Stationary distribution , probability flow analogy

In page rank we are multiplying Transpose Adjacency matrix with current page rank vector. If the result is stationary distribution it means it will converge and all values will stay same. We can think this as incoming probability to a node is equal to outgoing probability.


For detailed balance condition T(x'->x)P(x') = T(x->x')P(x) if the graph is undirected
So total flow of x' times fraction of flow to x T(x'->x) is equal to total flow of x times the fraction of flow from x to x' is equal means incoming and outgoing probability flow in the full duplex edge.

It is possible that, this concept is already originating from real physical phenomena such as fluid flow or air flow.

Friday, October 05, 2012

Bayesian F1

Decision theory considers  the posterior probability to decide which class to chose , or weight using a cost function. However in this setting its objective is the accuracy. Most often we are more interested in more balanced measures such as F1 due to skewed classes. How would be F1 maximizer Bayesian decision mechanism if the distribution is exactly known ? I guess this is already implemented in somewhere , however I didn't face it yet.

Thursday, October 04, 2012

Latex 2 column figure



\begin{figure*}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=2\columnwidth]{NAME}}
\caption{NAME}
\end{center}
\vskip -0.2in
\end{figure*}

Wednesday, October 03, 2012

More appealing

Novel, hierarchical , jointly learning , hybrid , semi supervised , hyper-parameters , large data , distributed , active , cost sensitive.

Wednesday, September 19, 2012

Training , Validation , Test set , A more general case

A regular practice is dividing training data into tree parts to evaluate the performance of the algorithm. Training set is used to train  parameters of the model. Validation set is used to "train" hyper parameters of the model and test set is used to evaluate the model.
     For more general sense what if I divide my training set into n part and use first set to bottom level parameters  for the tree (One can use topological sort in graphical model) of the parameters. Than iterate next upper level training upper level parameters in next set. At the end we would have no parameters above the top which corresponds to a test set. Basically , make the distinction between parameters and hyper parameters fuzzy.

Polynomial multiplication , base x

For representing polynomials we can use base notation , it will make some operation easier.
for example 1+x+x^2 = (111)_x

For multiplication (x+1) * (x+1) = x^2 + 2x + 1
We can represent as (11)_x + (11)_x = (121)_x
It becomes like a regular multiplication if it does not excess 10 for one digit , but if it does than we need to seperate number e.g. (1,13,1)_x = x^2 + 13x + 1
We can assume at this point x > R (all real numbers) in base notation.
Robot Particles Display:

Current display devices can show any image for a very cheap price. But it will be very intresting if someone tries to implement display using multiple robots. For example each robot will have fixed color and will stay in rectengular area. After input comes the positions of the robots will change to approximate the given image input. If this could be done in 2D in Earth. It could be extentable 3D in space since gravity no longer becomes a problem. That will result real 3D display using robotic particles. A good science finction scenario :).
It would be possible to describe everything scientifically, but it would make no sense; it would be without meaning, as if you described a Beethoven symphony as a variation of wave pressure.

Albert Einstein

Friday, September 14, 2012

Power Law

What is the relation between power law behavior and TF.Idf frequencies. They are supporting each other.

Saturday, September 08, 2012

Arbitrary Complexity PageRank-Prestige


Prestige :


A

Page Rank:

 

A + A_f 




Hub - Authority :


HITS algorithm - Incoming -> Authority , Outgoing -> Hub 

Arbitrary  :




Hub Authority + Unknown L class if pointed by Hubs the more the L it is and if A pointed by L the more the Authority it is (Or maybe having too many Hubs makes L strong). You can define anything.

The question is for a real world scoring could we find a optimum score graph given a large data ? Since no true score value exist for arbitrary model there is no criteria for evaluation yet.   





Thursday, August 30, 2012

KNN

       KNN comes from very basic idea. "Look at the closer samples", the science behind this classifier how it is proven to be useful. I mean , If you just come up with an heuristic which is not proven  it does not have big impact.

Sunday, August 12, 2012

Javascript Infinity loop


I don't want to forget this :

I used for loop in my recursive calls using i variable



function recursive(param)
{
//Some stuff
        for (i=0;i            recursive(otherparam)
}

It was creating  infinity loop. The problem that I realized is i is behaving like and global variable , thus creating an infinity loop.

Fix by :

for (var i=0;i</*A Value*/;i++)

Wednesday, August 01, 2012

http://en.wiktionary.org/wiki/mispelling mispelling Common misspelling of misspelling.

Sunday, April 22, 2012

Prime Graphs: There are some efficent algorithms for restricted set of graphs for NP hard problems. It gives me the intuition that there are core graph of those restricted set which divides all elements in their set (Altough division is not defined yet). In this sense there exist an algorithm which solves this restricted set efficiently by solving its core problem. These core's are called prime graphs which is analogous to prime numbers where prime number has their unique divisors. Division means that there is polynomial algorithm that can convert problem into solution or core problem. To solve those prime graphs an machine need to be started to solve and store them. Then anyone who needs a solution to their problem need to find a conversion to prime graph and query from already solved graphs. This is different from polynomial time conversion from one problem type to another. This time the iteration held on graphs as similar to numbers.

Friday, March 23, 2012

Ubuntu install VTK:

Required packages :

sudo apt-get install nvidia-current cmake libglu1-mesa-dev python-vtk g++ libxt-dev

Download VTK :

tar -zxvf vtk.tar.gz VTK
cmake VTK/.

I got error like iostream.h not found,
than I removed whole build directory and reopen archive again.
This time it seems working.

Tuesday, March 06, 2012

Friday, February 24, 2012

Old Casette Players

Nowadays no body using cassette player. I bought a cassette to listen music from my smartphone . It would be good if we can build an equipment to replace cassette player and allows user to connect his smartphone directly to this equipment. Than smartphone could be used as GPS, television or music player.

Wednesday, February 22, 2012

Wrong Normal Assumption on Linear Classifiers;

Normal assumption on weight vectors regularizes the by some precision matrix gamma. However normal distribution takes Mahalonobis distance , thus it fails to capture scaled version of weight vector.Also, If weight vectors assumed to be unit vectors than they will be distributed on unit hyper-sphere. It may worth to use von-misses distribution , but I think it will not differ too much.

Tuesday, January 24, 2012

Old value of the variable

It is very common to create another variable to store old value of the variable. There could be some option to have .old of each object that stores old value of that object. An object oriented language can have this property to save time while debugging and writing programs.

Monday, January 02, 2012

Comparative Debug;

Sometimes it is not clear which algorithm is best for your program. Thus , it will be useful if there is something that allows to compare two alternative codes easily. People mostly preferring comments to do this. But I think there should be some support in IDE. For example some key combination assigns code partition to first version of execution where another key combination assigns code partition to second version.

Karsilastirmali debug islemi IDE'ler tarafindan desteklenseydi faydali olurdu. Cogu zaman hangi kodun daha iyi oldugunu bilmiyoruz comment in comment out yaparak denemeler yapiyoruz. Bunun yerine IDE bazi tus kombinasyonlari ile degisik kod versiyonlari olusturup bunlari karsilikli calistirmaya izin verseydi guzel olurdu. Bu version control sistemlerindeki branch olayinin kucuk bir hali olarak dusunulebilir.

Sunday, December 25, 2011

Monday, December 19, 2011

ROC using perfcurve :

[X,Y,T,AUC] = perfcurve(labels,scores,posclass)

Perf curve returns points on ROC curve. To obtain more robust results using multiple iteration; labels and scores should be cell array containing multiple iteration labels. In this case it returns Mx3 matrix for X , Y , AUC , first column represents the mean value and others are bounds.

Wednesday, December 14, 2011

Kernighan's law:

Debugging is twice as hard as the code first place. Therefore , if you write the code as cleverly as possible , by definition not smart enough to debug it .

Friday, December 02, 2011

I wonder if we can represent irrational number using some finite primitive irrational numbers + rational numbers like in complex numbers (i is primitive).

Thursday, November 17, 2011

Is sun rotating around world or is world rotating around sun ?

It depends actually. Where did you define your origin ? Assume that I defined origin of universe as myself my position will be (0,0,0). So any rotation transformation will not change my position. It means all of the other object will turn around myself. Ofcourse it is intuitive to chose sun as origin in first sense , but it is not necessary , same transformations could be applied without chosing sun as an origin. There could be some standart system , than we can represent any position in universe by 3 coordinates. We can easily calculate using its distance and angle. That is actually what OPENGL is doing

Wednesday, November 09, 2011

Self Compacting DRAM:

Memory compaction is very expensive process. It is required to done in hardware. This case we can have assembly instruction that removes specified area in the memory. Than rest of the parts need to be shifted to empty portion. This requires shift by n circuit. If shift by n circuit could be done in cheap and fast way , we will not need any segmentation and page frames etc.

Tuesday, October 25, 2011

Eclipse Memory Problem:

When I am working on OpenGL project on Eclipse I faced a lot of problem due to memory. When javaw uses more than 256 mb it crashes and Eclipse shuts down immediately. When it is first opened it occupies 233 MB memory. There should be some configuration parameter to fix that. Does anyone knows it ?

This could be solution :
Weird Polygon Faces :

When I was translating my polygons in OpenGL , I realize that there is something wrong in my shape. It was due to depth configuration. I solved using :

glEnable(GL_DEPTH_TEST);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

It took 1 day to understand what is the real problem.

Monday, October 17, 2011

Real Time Transcription:

In TV channels I realize that text is a little bit coming late after the speech. So , I think they are using employees to transcript the sound. A computer can do it very fast. But the problem is sound is very noisy , computer transcription could not be accurate enough. One way to solve that using human employees. But human should be very careful to transcript. It makes it slow. Better way could be using multiple human annotators and computer together. In this setting multiple human annotators provides multiple transcription than using learning from multiple experts methods speech could be transcribed into fast and accurate text. People not necessary to precise as previous scenario. It could be good application for my research.

Best LLM Model ?

Offcourse trying to sell the race cars for everyone will not work.