Abstract—Deep with different layers are incorporated like

Abstract—Deep
learning is the new big fastest growing field in machine learning. It
can significantly change the way we see Artificial Intelligence and achieving
their goal. This paper gives survey on different software tools. For comparing
this software tools uses different types of deep learning algorithms and evaluate
the performance of the software tools. Deep learning methods have resulted in
significant performance in many applications and for that several software
tools have been developed to facilitate their implementation.

 

Keywords-Machine
learning, Deep learning etc.

We Will Write a Custom Essay Specifically
For You For Only $13.90/page!


order now

 

I. INTRODUCTION

 

Since 2006 a class of machine learning techniques
developed, where many layers of non-linear information proceeds and executed.
Deep learning methods have recently influenced several application domains in
industries and research domains, namely computer vision, speech recognition,
and nature language processing, where they have enjoyed significant performance
improvements.1 Deep learning architecture has different types of layers like
convolutional, fully connected and recurrent layers, with multiple layers of
nonlinear processing units depends on the problem.5 It also has supervised
and unsupervised learning of feature representation with different layers are incorporated
like low-level to high-level.1 Deep learning layers include the hidden layers
of artificial neural network. This architecture is trained with changing of
weights. As increased the popularity of deep learning software tools facilitate
efficient implementation of these tools. Deep learning having different
algorithms, so they try to optimize different aspects of development of deep
learning algorithm.

Section II gives brief introduction about Deep
learning. Section III gives detail of Tools for deep learning. Summary of deep
learning tools gives comparative study in section IV and finally conclusion is
given in section V.

 

I. DEEP LEARNING

                   Deep learning is a branch of
machine learning based on a set of algorithms that attempt to model high level
abstractions in data by using a deep graph with multiple processing layers,
composed of multiple linear and non-linear transformations.1 Some of the
representations in Deep learning are inspired by nervous system.6 Deep
learning is replacing hand-crafted measurable property with efficient
algorithms for unsupervised or semi-supervised feature learning and
hierarchical feature extraction. Each successive layer uses the output from the
previous layer as input. Deep learning include hidden layers of an artificial
neural network and sets of complicated proposi-tional formulas. Applications of
deep learning includes pattern analysis and classification.5

 

 

 

III. TOOLS FOR DEEP LEARNING

 

A. Pylearn2

 

Pylearn2 is one of the
deep learning software tool which is designed for flexibility and extensibility
of machine learning algorithms. Pylearn2 is implemented by LISA at University
de Montreal. Pylearn2 provides a efficient GPU convolutional network library
and Most of its functionality is built on top of Theano. That means Pylearn2
can write algorithms using mathematical expressions.2 Pylearn2 was used to
set the state of the art on MNIST, CIFAR-10, CIFAR-100, and SVHN.5

 

B. Torch

 

   Torch is machine learning library and used
as deep learning software tool. Torch is an open source library. Lua is the
scripting language is used. Because of torch has combination of flexibility and
computational efficiency is popular software tool for deep learning. As some
drawbacks in torch that reimplementation of logic cannot support, for that
facilitates rapid development new open source framework is implemented:
Torchnet. Torchnet provides terms key abstractions, boilerplate code and
reference implementation reusable and efficient.3

      
The goal of Torch is to have maximum flexibility and speed in building
your scientific algorithms while making the process extremely simple. Torch
comes with a large ecosystem of community-driven packages in machine learning,
computer vision, signal processing, parallel processing, image, video, audio
and networking among others, and builds on top of the Lua community.

 

Core
Features

     •    a powerful N-dimensional array

     •    lots of routines for indexing, slicing,
transposing, …

     •    amazing interface to C, via LuaJIT

     •    linear algebra routines , neural network
etc

     •    Fast and efficient GPU support

     •    Embeddable, with ports to iOS and Android
backends

C.
Theano

 

Theano is the deep learning tool that evaluate
mathematical expressions consist multi-dimensional arrays.2 It is Python
script language. As it involving mathematical expression, it performs
data-intensive calculation up to 140x faster than CPU. Theano provides a
language that actual implementation for describing expressions. Many
academic researchers in the field of deep learning rely on Theano, the
grand-daddy of deep-learning frameworks, which is written in Python. Theano is
a library that handles multidimensional arrays, like Numpy. Used with other
libs, it is well suited to data exploration and intended for research.

 

Core
Features

·        
Python
+  Multidimensional Array Support

·        
Computational
graph is nice abstraction

·        
RNNs
fit nicely in computational graph

·        
High
level wrappers (Keras, Lasagne) ease the pain

 

 

D. Caffe

 

Caffe is developed by the Berkeley Vision and
Learning Center(BVLC) created by Yangqing Jia and led by Evan Shelhamer. Caffe
is open source framework, C++ library with Python and MATLAB, and with CUDA for
GPU computation for deploying deep learning architecture.4 Caffe provides
multimedia framework for state-of-art deep learning algo-rithms. For cloud
environment developing prototype machines caffe allows platform switching.
Caffe provides a toolkit for training, deploying, testing, fine-tuning models
with examples.

 

Core
Features

·        
Good
for feed forward networks and image processing

·        
Good
for fine tuning existing networks

·        
Train
models without writing any code

·        
Python
interface is pretty useful

 

 

E. Cuda-convnet

 

Cuda-convnet is a fast neural network in C++, CUDA
and Python with implementation of convolutional neural networks. Convents have
lots of activation units and their connection. This is possible because in
Convnet neurons share weights between connection and therefore reduce the
number of parameters.8

 

F. Keras

 

            Keras is a deep-learning library
that sits a top Theano and TensorFlow, providing an intuitive API inspired by
Torch. Perhaps the best Python API in existence. Deeplearning4j relies on Keras
as its Python API and imports models from Keras and through Keras from Theano
and TensorFlow. It was created by Francois Chollet, a software engineer at
Google.

 

Core
Features

 

·        
Intuitive
API inspired by Torch

·        
Works
with Theano, TensorFlow and backends

·        
Fast
growing framework

·        
Likely
to become standard Python API for NNs

 

 

G.TensorFlow

 

H. MXNet

 

         MXNet offer powerful tools to help developers
exploit the full capabilities of GPUs and cloud computing. While these tools
are generally useful and applicable to any mathematical computation, MXNet places
a special emphasis on speeding up the development and deployment of large-scale
deep neural networks. MXNet supports two styles of programming: imperative programming (supported by the NDArray API) and symbolic programming (supported by the Symbol API) .In particular, MXNet offer the following
capabilities:

·        
Device
Placement: With MXNet,
it’s easy to specify where each data structures should live.

·        
Multi-GPU
training: MXNet makes
it easy to scale computation with number of available GPUs.

·        
Automatic
differentiation: MXNet automates
the derivative calculations that once bogged down neural network research.

·        
Optimized
Predefined Layers: While you can code up
your own layers in MXNet, the predefined layers are optimized for
speed, outperforming competing libraries.

 

 

 

IV. SUMMARY OF DEEP LEARNING TOOL

 

We have studied deep learning software tools in
details as shown in table I. Deep learning methods gives the significance
performance improvements in many application domains. Therefore, several deep
learning software tools are imple-mented.9 This paper represents comparative
study of deep learning tools. Comparative aspects considered like
Extensi-bility, Hardware utilization, Performance on single machine for CPU and
GPU setting, Architecture of tools, Ecosystem, Cross-Platform, Open source,
Interface, Modelling capability, etc.8 This paper has given comparative study
on the speed. Speed performance included the two type of methods: Forward time
and Gradient computation time. Torch and Theano are most easily extensible in
nature. In deep learning architecture Torch gives best performance on CPU,
followed by Theano. Other side Theano achieves best performance on GPU.
Cuda-convnet optimizes multiple objects.15

 

V. CONCLUSION

 

We studied eight Deep learning software tools in
detail and did the comparison of these tools.

 

REFERENCES

 

1  
S. Bahrampour, N. Ramakrishnan, L.
Schott, M. Shah, Research and Technology Center, Robert Bosch LLC Comparative
Study of Deep Learning Software Frameworks.

 

2  
Ian J. Goodfellow, David Warde-Farley,
Pascal Lamblin, Vincent Du-moulin, Mehdi Mirza, Razvan Pascanu, James Bergstra,
Frederic Bastien, and Yoshua Bengio, Departement dInformatique et de Recherche
Oper-ationelle, Universite de Montreal Pylearn2: a machine learning research
library.

 

3   
R. Collobert, Laurens van der Maaten, A.
Joulin, Facebook AI Research, 1 Hacker Way, Menlo Park CA 94025 / 770 Broadway,
New York NY 10003, USA Torchnet: An Open-Source Platform for (Deep) Learning
Research

 

4   
Yangqing
Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross
Girshick, Sergio Guadarrama, Trevor Darrell, SUBMITTED to ACM MULTIMEDIA 2014
OPEN SOURCE SOFTWARE COMPE-TITION, UC Berkeley EECS, Berkeley, CA 94702 Caffe:
Convolutional Architecture for Fast Feature Embedding

5  
Introduction to Deep learning
algorithm,Deep learning tutorials,http://deeplearning.net/tutorial/

6  
A beginners introduction to Deep
Learning, Article by Samvita,
http://hyperverge.co/a-beginners-introduction-to-deep-learning/

7  
Deep learning book, Chapter 1,
Introduction, http://www.deeplearningbook.org/contents/intro.html

 

8   
Cuda-convnet, High-performance C++/CUDA implementation
of convolution of neural networks, https://code.google.com/p/cuda-convnet/

9  
Evaluation of Deep Learning Toolkits,
https://github.com/zer0n/deepframeworks/performance

x

Hi!
I'm James!

Would you like to get a custom essay? How about receiving a customized one?

Check it out