JVM based DeepLearning on IoT data with Apache Spark
DeepLearning frameworks are popping up at very high frequency but only a few of them are suitable to run on clusters, use GPUs and supporting topologies beyond Feed-Forward at the same time. DeepLearning4J, ApacheSystemML and TensorSpark feature all this without forcing you to learn new exotic programming languages and in addition also scales-out on well established infrastructures like ApacheSpark. In this talk we will introduce DeepLearning4J on top of ApacheSpark with an example to create an anomaly detector for IoT sensor data with an LSTM auto encoder neural network. We will also explain how ApacheSystemML uses cost-based optimisers for Neural Network training and how TensorSpark parallelises TensorFlow on Apache Spark. Finally we will show how such systems are used in the IBM Watson Developer Cloud for Visual Recognition, Psychological Profiling and Document Classification.