Open Source Deep Learning Project: Deeplearning4J

Deeplearning4J: Neural Net Platform

Project Website: http://deeplearning4j.org/

Github Link: https://github.com/deeplearning4j/deeplearning4j

Description

Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J is designed to be used in business environments on distributed GPUs and CPUs. Skymind is its commercial support arm.

Deeplearning4J is an Apache 2.0-licensed, open-source, distributed neural net library written in Java and Scala.

Deeplearning4J integrates with Hadoop and Spark and runs on several backends that enable use of CPUs and GPUs. The aim is to create a plug-and-play solution that is more convention than configuration, and which allows for fast prototyping.

The most recent stable release in Maven Central is 0.4-rc3.9, and the current master is 0.4-rc3.10-SNAPSHOT. For more on working with snapshots, see this page.

Open Source Deep Learning Project: RNNLIB

RNNLIB

Project Website: https://sourceforge.net/projects/rnnl/

Github Link: None

Description

RNNLIB is a recurrent neural network library for sequence learning problems. Applicable to most types of spatiotemporal data, it has proven particularly effective for speech and handwriting recognition.

full installation and usage instructions given at
http://sourceforge.net/p/rnnl/wiki/Home/

Features
LSTM
Multidimensional recurrent neural networks
Connectionist temporal classification
Adaptive weight noise (stochastic variational inference)

Open Source Deep Learning Project: RNNLM

RNNLM – Tomas Mikolov’s Recurrent Neural Network based Language models Toolkit

Project Website: http://rnnlm.org/

Github Link: None

Description

RNNLM Toolkit

Neural network based language models are nowdays among the most successful techniques for statistical language modeling. They can be easily applied in wide range of tasks, including automatic speech recognition and machine translation, and provide significant improvements over classic backoff n-gram models. The ‘rnnlm’ toolkit can be used to train, evaluate and use such models.

The goal of this toolkit is to speed up research progress in the language modeling field. First, by providing useful implementation that can demonstrate some of the principles. Second, for the empirical experiments when used in speech recognition and other applications. And finally third, by providing a strong state of the art baseline results, to which future research that aims to “beat state of the art techniques” should compare to.

Open Source Deep Learning Project: DeepLearnToolbox

DeepLearnToolbox: Matlab/Octave toolbox for deep learning

Project Website: None

Github Link: https://github.com/rasmusbergpalm/DeepLearnToolbox

Description

Matlab/Octave toolbox for deep learning. Includes Deep Belief Nets, Stacked Autoencoders, Convolutional Neural Nets, Convolutional Autoencoders and vanilla Neural Nets. Each method has examples to get you started.

A Matlab toolbox for Deep Learning.

Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data. It is inspired by the human brain’s apparent deep (layered, hierarchical) architecture. A good overview of the theory of Deep Learning theory is Learning Deep Architectures for AI

For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.

The Next Generation of Neural Networks (Hinton, 2007)
Recent Developments in Deep Learning (Hinton, 2010)
Unsupervised Feature Learning and Deep Learning (Ng, 2011)
If you use this toolbox in your research please cite Prediction as a candidate for learning deep hierarchical models of data

@MASTERSTHESIS\{IMM2012-06284,
author = “R. B. Palm”,
title = “Prediction as a candidate for learning deep hierarchical models of data”,
year = “2012”,
}
Contact: rasmusbergpalm at gmail dot com

Open Source Deep Learning Project: Chainer

Chainer: A Powerful, Flexible, and Intuitive Framework of Neural Networks

Project Website: http://chainer.org/

Github Link: https://github.com/pfnet/chainer

Description

Chainer is a flexible framework for neural networks. One major goal is flexibility, so it must enable us to write complex architectures simply and intuitively.

Most existing deep learning frameworks are based on the “Define-and-Run” scheme. That is, first a network is defined and fixed, and then the user periodically feeds it with mini-batches. Since the network is statically defined before any forward/backward computation, all the logic must be embedded into the network architecture as data. Consequently, defining a network architecture in such systems (e.g. Caffe) follows a declarative approach. Note that one can still produce such a static network definition using imperative languages (e.g. Torch7 and Theano-based frameworks).

In contrast, Chainer adopts a “Define-by-Run” scheme, i.e., the network is defined on-the-fly via the actual forward computation. More precisely, Chainer stores the history of computation instead of programming logic. This strategy enables to fully leverage the power of programming logic in Python. For example, Chainer does not need any magic to introduce conditionals and loops into the network definitions. The Define-by-Run scheme is the core concept of Chainer. We will show in this tutorial how to define networks dynamically.

Open Source Deep Learning Project: Keras

Keras: Deep Learning library for Theano and TensorFlow

Project Website: http://keras.io/

Github Link: https://github.com/fchollet/keras

Description

Keras is a minimalist, highly modular neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.

Use Keras if you need a deep learning library that:

allows for easy and fast prototyping (through total modularity, minimalism, and extensibility).
supports both convolutional networks and recurrent networks, as well as combinations of the two.
supports arbitrary connectivity schemes (including multi-input and multi-output training).
runs seamlessly on CPU and GPU.
Read the documentation at Keras.io.

Keras is compatible with: Python 2.7-3.5.

Open Source Text Processing Project: CMU Flite

CMU Flite: a small, fast run time synthesis engine

Project Website: http://www.festvox.org/flite/
Github Link: None

Description

CMU Flite (festival-lite) is a small, fast run-time open source text to speech synthesis engine developed at CMU and primarily designed for small embedded machines and/or large servers. Flite is designed as an alternative text to speech synthesis engine to Festival for voices built using the FestVox suite of voice building tools.
CMU Flite 2.0.0-release is now released as open source. Flite offers:

C only, very portable source
Thread safe (really this time)
Multi-voice, multi-language
Support for converting the latest Clustergen Random Forest voices for CMU Flite
Demos in English and Indic
New in this version

Loadable voices (13 US English, in various accents)
Android support
INDIC language support ( Hindi, Tamil, Telugu and Marathi)
Here are slides about Flite from an early talk given by AWB slides.ps slides.pdf

Here is a recent publication at the 4th ISCA Speech Synthesis Workshop by Alan W Black and Kevin A. Lenzo html, postscript

Open Source Text Processing Project: MaryTTS

MaryTTS: The MARY Text-to-Speech System

Project Website: http://mary.dfki.de/
Github Link: None

Description

MaryTTS is an open-source, multilingual Text-to-Speech Synthesis platform written in Java. It was originally developed as a collaborative project of DFKI’s Language Technology Lab and the Institute of Phonetics at Saarland University. It is now maintained by the Multimodal Speech Processing Group in the Cluster of Excellence MMCI and DFKI.

As of version 5.2-SNAPSHOT, MaryTTS supports German, British and American English, French, Italian, Swedish, Russian, Turkish, and Telugu; more languages are in preparation. MaryTTS comes with toolkits for quickly adding support for new languages and for building unit selection and HMM-based synthesis voices.

Four parts of the TTS system can be distinguished:

the preprocessing or text normalisation;
the natural language processing, doing linguistic analysis and annotation;
the calculation of acoustic parameters, which translates the linguistically annotated symbolic structure into a table containing only physically relevant parameters;
and the synthesis, transforming the parameter table into an audio file.

Open Source Deep Learning Project: Lasagne

Lasagne: Lightweight library to build and train neural networks in Theano

Project Website: http://lasagne.readthedocs.io/

Github Link: https://github.com/Lasagne/Lasagne

Description

Lasagne is a lightweight library to build and train neural networks in Theano. Its main features are:

Supports feed-forward networks such as Convolutional Neural Networks (CNNs), recurrent networks including Long Short-Term Memory (LSTM), and any combination thereof
Allows architectures of multiple inputs and multiple outputs, including auxiliary classifiers
Many optimization methods including Nesterov momentum, RMSprop and ADAM
Freely definable cost function and no need to derive gradients due to Theano’s symbolic differentiation
Transparent support of CPUs and GPUs due to Theano’s expression compiler
Its design is governed by six principles:

Simplicity: Be easy to use, easy to understand and easy to extend, to facilitate use in research
Transparency: Do not hide Theano behind abstractions, directly process and return Theano expressions or Python / numpy data types
Modularity: Allow all parts (layers, regularizers, optimizers, …) to be used independently of Lasagne
Pragmatism: Make common use cases easy, do not overrate uncommon cases
Restraint: Do not obstruct users with features they decide not to use
Focus: “Do one thing and do it well”

Open Source Deep Learning Project: Caffe

Caffe: Deep learning framework by the BVLC

Project Website: http://caffe.berkeleyvision.org/

Github Link: https://github.com/BVLC/caffe

Description

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

Why Caffe?
Expressive architecture encourages application and innovation. Models and optimization are defined by configuration without hard-coding. Switch between CPU and GPU by setting a single flag to train on a GPU machine then deploy to commodity clusters or mobile devices.

Extensible code fosters active development. In Caffe’s first year, it has been forked by over 1,000 developers and had many significant changes contributed back. Thanks to these contributors the framework tracks the state-of-the-art in both code and models.

Speed makes Caffe perfect for research experiments and industry deployment. Caffe can process over 60M images per day with a single NVIDIA K40 GPU*. That’s 1 ms/image for inference and 4 ms/image for learning. We believe that Caffe is the fastest convnet implementation available.

Community: Caffe already powers academic research projects, startup prototypes, and even large-scale industrial applications in vision, speech, and multimedia. Join our community of brewers on the caffe-users group and Github.