DeepDSL is a domain specific language (DSL) embedded in Scala for writing deep learning network applications.
As promised, we are now very happy to announce a major milestone release: DeepDSL v1.0. Also another very important decision we make is to release the core Scala source code for reference in the hope that the system will become more popular and also facilitate advanced deep learning with more transparency. Please refer our papers below if you are using our code in academic research: - "DeepDSL: A Compilation-based Domain-Specific Language for Deep Learning" in proceedings of the International Conference on Learning Representations (ICLR 2017), Toulon, France. [pdf DeepDSL] - "Design and Implementation of DeepDSL: A DSL for Deep Learning", Computer Languages, Systems & Structures, Volume 54, December 2018, Pages 39-70. [https://www.sciencedirect.com/science/article/pii/S1477842418300319]
Please read our LICENSE file and contact [email protected] for commercial development and cooperation.
DeepDSL is a domain specific language (DSL) embedded in Scala for writing deep learning network applications.
Below we list the runtime and memory performance comparison between DeepDSL, Tensorflow, and Caffe with a single Nvidia Tesla K40c GPU.
Please refer to our paper draft DeepDSL for full details.
cudnn64_5.dllhas to be added to the
PATHenvironment variable. On Linux, the path that contains the
libcudnn.so.5has be added to the
LD_LIBRARY_PATH
After the previous Maven Build step, you can cd to the deepdsl-java folder and run the following based on your operating system:
There are two util Python scripts under the folder src/main/python (both should be run from the deepdsl project root folder).
dataset/mnistfolder
python ../src/main/python/mnist_data_handler.py(called in the
deepdsl-javadirectory). This will pull and extract the mnist dataset to
dataset/mnist
python src/main/python/imagenet_data_selector.pyand then follow the on-screen instructions to apply the desired parameters and run again
python imagenet_data_selector.py ~/data/ILSVRC2012_img_train ~/data/temp 5 50 0.3 0.2, which selects from 5 categories (50 images per catetory) from ~/data/ILSVRC2012imgtrain folder and stores selected images to ~/data/temp folder, where 30% are stored as validation dataset and 20% are stored as test dataset
Each program assumes a location for the training and test data.
examples/imagenet/create_imagenet.shto create the lmdb data from the original Imagenet dataset. Please hang tight, we are adding our scripts soon so you don't have to resort to outside reources.
Each program will save the trained model, the set of trained parameters (as serialized Java objects), into a default directory.
You can generate Java source for a particular network by running the Scala test program TestNetwork.scala. While this is a Scala program, you can run it as a JUnit test to generate Java source code, the generated code will be written to src/main/java/deepdsl/gen/. You can run this directly from IDE, or cd to deepdsl-java folder and run from command line as the below after modifying the code and executing Maven build.
val K = 10 // # of classes val N = 500; val C = 1; val N1 = 28; val N2 = 28 // batch size, channel, and x/y size// Specifying train dataSet. (code gen will also use this to find test dataSet) val y = Vec._new(Mnist, "label", "Y", N) val x = Vec._new(Mnist, "image", "X", N, C, N1, N2) // followings are tensor functions val cv1 = CudaLayer.convolv("cv1", 5, 20) // convolution layer with kernel 5, stride 1, padding 0, and output channel 20 val cv2 = CudaLayer.convolv("cv2", 5, 50) val mp = CudaLayer.max_pool(2) // max pooling with kernel 2 and stride 2 val flat = Layer.flatten(4, 1) // flatten a 4-D tensor to 2-D: axis 0 - 3 becomes axis 0 and axis 1-3 val f = Layer.full("fc1", 500) // fully connected layer with output dimension 500 val f2 = Layer.full("fc2", K) val softmax = CudaLayer.softmax val relu = CudaLayer.relu(2) // ReLU activation function (2-D) // o is a left-associative function composition operator: f o g o h == (f o g) o h val network = f2 o relu o f o flat o mp o cv2 o mp o cv1 println(typeof(network)) // typecheck the network and print out the tensor function type val x1 = x.asCuda // load x (images) to GPU memory val y1 = y.asIndicator(K).asCuda // convert y (labels) to indicator vectors and load into GPU memory val c = (Layer.log_loss(y1) o softmax o network) (x1) // represent the log-loss of the training data val p = (Layer.precision(y1) o network) (x1) // represent the accuracy of the test data val param = c.freeVar.toList // discover the list of training parameters // parameters: name, training iterations, test iterations, learn rate, momentum, weight decay, cropping (0 means none) val solver = Train("lenet", 100, 10, 0.01f, 0.9f, 0.0005f, 0) val loop = Loop(c, p, (x, y), param, solver) // represent the training and testing loop runtimeMemory(loop.train) // print out the detailed memory consumption for one training loop parameterMemory(loop) // print out the parameter memory use workspaceMemory(loop.train) // print out the GPU (convolution) workspace use (only if you has Nvidia GPU) cudnn_gen.print(loop) // generate Java source code
dataset/imagenet/ilsvrc12_train_lmdb.
I can run the generated code, such as Alexnet128, however, I received exception as shown below when the execution finishes, what am I supposed to do?
java.io.FileNotFoundException: src/main/java/deepdsl/gen/Alexnet128/cv1_B.ser (No such file or directory) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at java.io.FileOutputStream.(FileOutputStream.java:101)
......
java.io.FileNotFoundException: src/main/java/deepdsl/gen/Alexnet128/cv1_W.ser (No such file or directory) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at java.io.FileOutputStream.(FileOutputStream.java:101)
......
src/main/java/deepdsl/gen/alexnet� before you run the program, otherwise you lose it. Please refer to the Default location for trained parameters section for more details.
[dataset/imagenet/]: <>