Object Classification on aYahoo image dataset

Disclaimer:
1. You need to have Caffe installed on your system to run the below tasks. If not follow the tutorial here to get Caffe up and running on your system.
2. You need to download aYahoo image dataset here.
3. You need to get all the pre-trained models: alexnet.caffemodel, coffenet.caffemodel and googlenet.caffemodel to get started on extracting features from images.

We will start by getting the paths to all the downloaded images into a text file. Run this line in terminal with correct path to images’ folder and path to text file in which we need to append all the paths to images.

find `pwd`/examples/images/ayahoo -type f -exec echo {} \; > examples/tempFeaExtr/temp.txt

Now we need to label the images. I am assigning labels starting from 0. It is recommended to sort all the names in text file, so we can divide the whole dataset into train and test datasets with reasonable number of images of each class in both datasets rather than randomly the whole data.

#to_label
outfile= open('zz_temp.txt', 'w')
infile= open('z_temp.txt', 'r')
myset= []

for line in infile:
 a= line[(len(line)-(line[::-1].index('/'))):-1]
 a= a[:a.index('_')]
 myset.append(a)

 if a== "bag":
 b= "0"
 elif a== "building":
 b= "1"
 elif a== "carriage":
 b= "2"
 elif a== "centaur":
 b= "3"
 elif a== "donkey":
 b= "4"
 elif a== "goat":
 b= "5"
 elif a== "jetski":
 b= "6"
 elif a== "monkey":
 b= "7"
 elif a== "mug":
 b= "8"
 elif a== "statue":
 b= "9"
 elif a== "wolf":
 b= "10"
 elif a== "zebra":
 b= "11"

 outfile.write(line[:-1]+ " "+ b+ "\n")
print(set(myset))

infile.close()
outfile.close()

#to_sort
infile= open('zz_temp.txt', 'r')
outfile= open('z_temp.txt', 'w')
myset= []

for line in infile:
 myset.append(line[:-1])

myset.sort()
for a in myset:
 outfile.write(a+"\n")

infile.close()
outfile.close()

As we have labeled and sorted the whole database, now, we divide it into two sets namely train_data.txt and test_data.txt. Try to keep the ratio of “no. of train images/ no. of test images” constant for all the classes. Now, we have train_data and test_data to proceed on to next step.

I am going to extract features of every image using the pre-trained models. After extracting I am gonna train SVM classifiers to predict the classes of test_data.

Link to the code to extract features and to train SVMs id here on GitHub.

Happy classifying images.
Adios

Advertisements

Install Caffe on Ubuntu 14.04

Disclaimer:
1. As my machine has no GPU hardware, I am going to install Caffe without CUDA support.
2. I am going to install only Python wrapper for Caffe.

As the installation page on Caffe has no detailed instructions to install all the dependencies required and get your system ready to run CNNs , I am writing this small tutorial to set Caffe Up and Running on your machine.

Firstly, we will get stared by downloading latest release of Caffe from github. (You need to have Git installed on your machine)

git clone https://github.com/BVLC/caffe.git
#add these following lines in the end of your .bashrc file
export PYTHONPATH=$PYTHONPATH:$HOME/caffe/python
export PYTHONPATH=$PYTHONPATH:$HOME/caffe/python/caffe

It is advised to use Anaconda Python distribution as it installs most of the requirements for python wrapper around Caffe (though it install a lot of packages we have no use). But, installing Anaconda is up to you.
Download Anaconda and run shell script. Add the path of bin to #PATH in file ~/.bashrc.

bash Anaconda-2.1.0-Linux-x86_64.sh
#add the following line at the end to your .bashrc
export PATH=/home/suryateja/anaconda/bin:$PATH

Get gflags and install on your system.

wget https://github.com/schuhschuh/gflags/archive/master.zip
unzip master.zip
cd gflags-master
mkdir build && cd build
export CXXFLAGS="-fPIC" && cmake .. && make VERBOSE=1</pre>
<pre>make
sudo make install

Download google-glog and install. You can also install through apt-get. And install snappy.

tar -xvzf google-glog_0.3.3.orig.tar.gz
cd glog-0.3.3/
./configure
make
sudo make install
agi libgoogle-glog-dev
agi libsnappy-dev

Get leveldb onto your machine and lmdb with its dependencies.

git clone https://code.google.com/p/leveldb/
cd leveldb
make
cp --preserve=links libleveldb.* /usr/local/lib
sudo cp --preserve=links libleveldb.* /usr/local/lib
sudo cp -r include/leveldb /usr/local/include/
ldconfig
sudo ldconfig
sudo apt-get install libffi-dev python-dev build-essential
sudo apt-get install liblmdb-dev

Download Atlas and Lapack (directly downloads when you click on link!). But before installing, you need to modify your CPU frequency/scaling. You you get errors saving the file, try to use different editor. I have tried sublime, gedit. Finally with I can modify with emacs. Change the scaling for all of your CPU cores to replace the single word in the file with ‘performance’.

gksu emacs /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
gksu emacs /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
gksu emacs /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
gksu emacs /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

bunzip2 -c atlas3.10.2.tar.bz2 | tar xfm -
mv ATLAS ATLAS3.10.2
cd ATLAS3.10.2
mkdir Linux_C2D64SSE3
cd Linux_C2D64SSE3
../configure -b 64 -D c -DPentiumCPS=2400 --prefix=/home/suryateja/lib/atlas --with-netlib-lapack-tarfile=/home/suryateja/Downloads/lapack-3.5.0.tgz

make build
make check
make ptcheck
make time
sudo make install

Install dependencies for python wrapper if you didn’t choose to install Anaconda Python distrubution. The dependencies file is in caffe/python. And install hdf5 via apt-get.

cd python/
for req in $(cat requirements.txt); do sudo pip install $req; done
sudo apt-get install libhdf5-serial-dev

Download latest 3.0.0-beta version (file automatically downloads if you click) of OpenCV and build it. But before install all the dependencies of OpenCV.

sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

cd Downloads
cd opencv-3.0.0-beta/
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make
sudo make install

Now, we are almost done with getting all the dependencies of Caffe. So, lets dive into getting Caffe up and running on our machines. Modify the makefile as your needs as shown in installation page.

cd caffe
cp Makefile.config.example Makefile.config
make all
make test
make runtest

If you have errors while installing even followed this tutorial, you can search for any known issues on their issues page. One of the errors I faced is this. And I am able to solve it as suggested in the comments section of issue page.

At last, now, you have Caffe installed and running on your machine.
Happy training Deep Neural Networks. 😀