Abstract—Deep

learning is the new big fastest growing field in machine learning. It

can significantly change the way we see Artificial Intelligence and achieving

their goal. This paper gives survey on different software tools. For comparing

this software tools uses different types of deep learning algorithms and evaluate

the performance of the software tools. Deep learning methods have resulted in

significant performance in many applications and for that several software

tools have been developed to facilitate their implementation.

Keywords-Machine

learning, Deep learning etc.

I. INTRODUCTION

Since 2006 a class of machine learning techniques

developed, where many layers of non-linear information proceeds and executed.

Deep learning methods have recently influenced several application domains in

industries and research domains, namely computer vision, speech recognition,

and nature language processing, where they have enjoyed significant performance

improvements.1 Deep learning architecture has different types of layers like

convolutional, fully connected and recurrent layers, with multiple layers of

nonlinear processing units depends on the problem.5 It also has supervised

and unsupervised learning of feature representation with different layers are incorporated

like low-level to high-level.1 Deep learning layers include the hidden layers

of artificial neural network. This architecture is trained with changing of

weights. As increased the popularity of deep learning software tools facilitate

efficient implementation of these tools. Deep learning having different

algorithms, so they try to optimize different aspects of development of deep

learning algorithm.

Section II gives brief introduction about Deep

learning. Section III gives detail of Tools for deep learning. Summary of deep

learning tools gives comparative study in section IV and finally conclusion is

given in section V.

I. DEEP LEARNING

Deep learning is a branch of

machine learning based on a set of algorithms that attempt to model high level

abstractions in data by using a deep graph with multiple processing layers,

composed of multiple linear and non-linear transformations.1 Some of the

representations in Deep learning are inspired by nervous system.6 Deep

learning is replacing hand-crafted measurable property with efficient

algorithms for unsupervised or semi-supervised feature learning and

hierarchical feature extraction. Each successive layer uses the output from the

previous layer as input. Deep learning include hidden layers of an artificial

neural network and sets of complicated proposi-tional formulas. Applications of

deep learning includes pattern analysis and classification.5

III. TOOLS FOR DEEP LEARNING

A. Pylearn2

Pylearn2 is one of the

deep learning software tool which is designed for flexibility and extensibility

of machine learning algorithms. Pylearn2 is implemented by LISA at University

de Montreal. Pylearn2 provides a efficient GPU convolutional network library

and Most of its functionality is built on top of Theano. That means Pylearn2

can write algorithms using mathematical expressions.2 Pylearn2 was used to

set the state of the art on MNIST, CIFAR-10, CIFAR-100, and SVHN.5

B. Torch

Torch is machine learning library and used

as deep learning software tool. Torch is an open source library. Lua is the

scripting language is used. Because of torch has combination of flexibility and

computational efficiency is popular software tool for deep learning. As some

drawbacks in torch that reimplementation of logic cannot support, for that

facilitates rapid development new open source framework is implemented:

Torchnet. Torchnet provides terms key abstractions, boilerplate code and

reference implementation reusable and efficient.3

The goal of Torch is to have maximum flexibility and speed in building

your scientific algorithms while making the process extremely simple. Torch

comes with a large ecosystem of community-driven packages in machine learning,

computer vision, signal processing, parallel processing, image, video, audio

and networking among others, and builds on top of the Lua community.

Core

Features

• a powerful N-dimensional array

• lots of routines for indexing, slicing,

transposing, …

• amazing interface to C, via LuaJIT

• linear algebra routines , neural network

etc

• Fast and efficient GPU support

• Embeddable, with ports to iOS and Android

backends

C.

Theano

Theano is the deep learning tool that evaluate

mathematical expressions consist multi-dimensional arrays.2 It is Python

script language. As it involving mathematical expression, it performs

data-intensive calculation up to 140x faster than CPU. Theano provides a

language that actual implementation for describing expressions. Many

academic researchers in the field of deep learning rely on Theano, the

grand-daddy of deep-learning frameworks, which is written in Python. Theano is

a library that handles multidimensional arrays, like Numpy. Used with other

libs, it is well suited to data exploration and intended for research.

Core

Features

·

Python

+ Multidimensional Array Support

·

Computational

graph is nice abstraction

·

RNNs

fit nicely in computational graph

·

High

level wrappers (Keras, Lasagne) ease the pain

D. Caffe

Caffe is developed by the Berkeley Vision and

Learning Center(BVLC) created by Yangqing Jia and led by Evan Shelhamer. Caffe

is open source framework, C++ library with Python and MATLAB, and with CUDA for

GPU computation for deploying deep learning architecture.4 Caffe provides

multimedia framework for state-of-art deep learning algo-rithms. For cloud

environment developing prototype machines caffe allows platform switching.

Caffe provides a toolkit for training, deploying, testing, fine-tuning models

with examples.

Core

Features

·

Good

for feed forward networks and image processing

·

Good

for fine tuning existing networks

·

Train

models without writing any code

·

Python

interface is pretty useful

E. Cuda-convnet

Cuda-convnet is a fast neural network in C++, CUDA

and Python with implementation of convolutional neural networks. Convents have

lots of activation units and their connection. This is possible because in

Convnet neurons share weights between connection and therefore reduce the

number of parameters.8

F. Keras

Keras is a deep-learning library

that sits a top Theano and TensorFlow, providing an intuitive API inspired by

Torch. Perhaps the best Python API in existence. Deeplearning4j relies on Keras

as its Python API and imports models from Keras and through Keras from Theano

and TensorFlow. It was created by Francois Chollet, a software engineer at

Google.

Core

Features

·

Intuitive

API inspired by Torch

·

Works

with Theano, TensorFlow and backends

·

Fast

growing framework

·

Likely

to become standard Python API for NNs

G.TensorFlow

H. MXNet

MXNet offer powerful tools to help developers

exploit the full capabilities of GPUs and cloud computing. While these tools

are generally useful and applicable to any mathematical computation, MXNet places

a special emphasis on speeding up the development and deployment of large-scale

deep neural networks. MXNet supports two styles of programming: imperative programming (supported by the NDArray API) and symbolic programming (supported by the Symbol API) .In particular, MXNet offer the following

capabilities:

·

Device

Placement: With MXNet,

it’s easy to specify where each data structures should live.

·

Multi-GPU

training: MXNet makes

it easy to scale computation with number of available GPUs.

·

Automatic

differentiation: MXNet automates

the derivative calculations that once bogged down neural network research.

·

Optimized

Predefined Layers: While you can code up

your own layers in MXNet, the predefined layers are optimized for

speed, outperforming competing libraries.

IV. SUMMARY OF DEEP LEARNING TOOL

We have studied deep learning software tools in

details as shown in table I. Deep learning methods gives the significance

performance improvements in many application domains. Therefore, several deep

learning software tools are imple-mented.9 This paper represents comparative

study of deep learning tools. Comparative aspects considered like

Extensi-bility, Hardware utilization, Performance on single machine for CPU and

GPU setting, Architecture of tools, Ecosystem, Cross-Platform, Open source,

Interface, Modelling capability, etc.8 This paper has given comparative study

on the speed. Speed performance included the two type of methods: Forward time

and Gradient computation time. Torch and Theano are most easily extensible in

nature. In deep learning architecture Torch gives best performance on CPU,

followed by Theano. Other side Theano achieves best performance on GPU.

Cuda-convnet optimizes multiple objects.15

V. CONCLUSION

We studied eight Deep learning software tools in

detail and did the comparison of these tools.

REFERENCES

1

S. Bahrampour, N. Ramakrishnan, L.

Schott, M. Shah, Research and Technology Center, Robert Bosch LLC Comparative

Study of Deep Learning Software Frameworks.

2

Ian J. Goodfellow, David Warde-Farley,

Pascal Lamblin, Vincent Du-moulin, Mehdi Mirza, Razvan Pascanu, James Bergstra,

Frederic Bastien, and Yoshua Bengio, Departement dInformatique et de Recherche

Oper-ationelle, Universite de Montreal Pylearn2: a machine learning research

library.

3

R. Collobert, Laurens van der Maaten, A.

Joulin, Facebook AI Research, 1 Hacker Way, Menlo Park CA 94025 / 770 Broadway,

New York NY 10003, USA Torchnet: An Open-Source Platform for (Deep) Learning

Research

4

Yangqing

Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross

Girshick, Sergio Guadarrama, Trevor Darrell, SUBMITTED to ACM MULTIMEDIA 2014

OPEN SOURCE SOFTWARE COMPE-TITION, UC Berkeley EECS, Berkeley, CA 94702 Caffe:

Convolutional Architecture for Fast Feature Embedding

5

Introduction to Deep learning

algorithm,Deep learning tutorials,http://deeplearning.net/tutorial/

6

A beginners introduction to Deep

Learning, Article by Samvita,

http://hyperverge.co/a-beginners-introduction-to-deep-learning/

7

Deep learning book, Chapter 1,

Introduction, http://www.deeplearningbook.org/contents/intro.html

8

Cuda-convnet, High-performance C++/CUDA implementation

of convolution of neural networks, https://code.google.com/p/cuda-convnet/

9

Evaluation of Deep Learning Toolkits,

https://github.com/zer0n/deepframeworks/performance