When building deep learning models, it is often required to check the model for consistency and proper parameters definition. In ANNdotNET, ml network models are designed using Visual Network Designer (VND), so it is easy to see the network configuration. Beside VND, in ANNdotNET there are several visualization features on different level: network preparation, model training phase, post training evaluation, performance analysis, and export results. In this blog post we will learn how to use those features when working with deep learning models
Visualization during network preparation and model training
When preparing network and training parameters, we need information about data sets, input format and output type. This information is relevant for selecting what type of network model to configure, what types of layers we will use, and what learner to select. For example the flowing image shows network configuration containing of 2 embedding layers, 3 dense layers and 2 dropout layers. This network configuration is used to train CNTK model for mushroom data set. As can be seen network layers are arranged as listbox items, and the user has possibility to see, on the highest level, how neural networks looks like, which layers are included in the network, and how many dimensions each layer is defined. This is very helpful, since it provides the way of building network very quickly and accurately, and it requires much less times in comparisons to use traditional way of coding the network in python, or other programming language.
ANNdotNET Network Settings page provides pretty much information about the network, input and output layers, what data set are defined, as well as whole network configuration arranged in layers. Beside network related information, the Network Settings tab page also provides the learning parameters for the network training. More about Visual Network Designer the ready can find on one of the previous blog post.
Since ANNdotNET implements MLEngine which is based on CNTK, so all CNTK related visualization features could be used. The CNTK library provides rich set of visualizations. For example you can use Tensorboard in CNTK for visualization not just computational graph, but also training history, model evaluation etc. Beside Tensorboard, CNTK provides logger module which uses Graphviz tool for visualizing network graph. The bad news of this is that all above features cannot be run on C#, since those implementation are available only in python.
This is one of the main reason why ANNdotNET provides rich set of visualizations for .NET platform. This includes: training history, model evaluation for training and validation data set, as well as model performance analysis. The following image show some of the visualization features: the training history (loss and evaluation) of minibatches during training of mushroom model:
Moreover, the following image shows evaluation of training and validation set for each iteration during training:
Those graphs are generated during training phase, so the user can see what is happening with the model. This is of tremendous help, when deciding when to stop the training process, or are training parameters produce good model at all, or this can be helpful in case when can stop and change parameters values. In case we need to stop the training process immediately, ANNdotNET provides Stop command which stops training process at any time.
Model performance visualization
Once the model is trained, ANNdotNET provides performance analysis tool for all three types of ML problems: regression, binary and multi class classification.
Since the mushrooms project is binary ML problem the following image shows the performance of the trained model:
Using Graphviz to visualize CNTK network graph in C#
We have seen that ANNdotNET provides all types of visualizations CNTK models, and those features are provided by mouse click through the GUI interfaces. One more feature are coming to ANNdotNET v1.1 which uses Grpahviz to visualize CNTK network graph. The feature is implemented based on original CNTK python implementation with some modification and style.
In order to use Graphviz to visualize network computation graph the following requirements must be met:
- Install Graphviz on you machine.
- Register Graphviz path as system variable. (See image below)
Now that you have install Graphviz tool, you can generate nice image of your network model directly in ANNdotNET just by click on Graph button above the Visual Network Designer (see image 1).
Here is some of nice graphs which can be generate from ANNdotNET preclaculated models.
In case you like this nice visualization features go to http://github.com/bhrnjica/anndotnet, download the latest version from release section or just download the source code and try it with Visual Studio, but don’t forget to give a star.
In the next blog post I will show you how visualization of CNTK computational graph is implemented, so you will be able to use it in your custom solutions.
Bahrudin Hrnjica Blog 
Hello, my system is windows 7 64 bits and ANNdotNET does not work, I would like to know why, my email is jatunyay.ssgg@gmail.com.
Hi
ANNdotNET is tested only on Windows 10 x64.
You can try to run on WIndows 7, but there are some requirements:
1. First the OS must be 64x bit.
2. Download software requirements fro this page: https://github.com/bhrnjica/anndotnet
3. Install it and try.
As I mentioned, the ANNdotNET is officially tested only on Windows 10 64 bit.
I have tested ANNdotNET v1.1 , and I could successfully run the application on Windows 7 sp1 x64.
Prior to run I had to install software that are specified in the main page of the project:
– .NET Framework 4.7.2, from here: http://go.microsoft.com/fwlink/?linkid=863262
– Visual C++ 2017 version 15.4 v14.11 toolset from here: https://aka.ms/vs/15/release/vc_redist.x64.exe
– Visual C++ Redistributable Packages for Visual Studio 2013 only vcredist_x64.exe file: from here https://www.microsoft.com/en-us/download/details.aspx?id=40784
So you should have no problems with running once you installed above prerequisites.
I will try and confirm, greetings.
Hello, install windows 64 64 bits according to the instructions and it did not work, when you install windows 10 with office 2019 if it works, I would like to make some suggestions:
1. NNdotNET could use genetic programming to optimize the adjustable parameters.
2. NNdotNET could work with more than 2 numerical or categorical outputs (Label), as it happens in the real world.
A query: when NNdotNET generates the excel file with the formula in a cell, by prefixing the = sign, excel does not accept, what can I be doing wrong?
thanks for your help
Framework
jatunyay.ssgg@gmail.com
Hi
Thanks for feed back,
1. ANNdotNET currently not support optimizing hyper-parameters with GA, we will concider it to put in the future.
2. ANNdotNET currently supports only one output variable (numeric or categorica). Definitely, more than one output will be implemented in the future release.
3. In order to run model in Excel, you should install ANNdotNET Excel addin. The installation instruction can be found at https://github.com/bhrnjica/anndotnet/blob/master/README.md
Hello, when you unzip ANNdotNETv1.1-rc20181115-rev1.zip there are no excel files anndotnet.exceladdin-AddIn64.xll and anndotnet.exceladdin-AddIn64-packed.xll
Thanks you for the feedback. I have included Excel Addin into installation package.
Please go to Release section (https://github.com/bhrnjica/anndotnet/releases/tag/v1.1) and download ANNdotNETv1.1-rev01.zip .
The zip files also included Excel Addin which you can install on your machine.
Please don’t hesitate to post comment in case you have any additional problem, and also if you have successfully install the Addin..
Hello, when you unzip ANNdotNET does not appear the folder bin to edit the system variable on the way, he can configure and complement the add-ons of excel anndotnet.exceladdIn-AddIn64, but he does not accept me the follow =
The path surrounded with red rectangle on the image is the path on my developer machine, and shows as an Example where the ANNdotNETv1.1-rev01.zip file should be unzipped.
For example: Assume you unzip the file at: c:\ANNdotNET folder.
The path you need to register is “c:\ANNdotNET” .
Hello, install in C: \ ANNdotNET and in system variables configure C: \ Program Files \ ANNdotNET \ also in excel add-ins configure anndotnet.exceladdIn-AddIn64, after restarting windows 10 place the = sign and it does not work, what can I do? be doing wrong?, I could send images to an email for more detail or vice versa, thanks for your support
In order to used ANNdotNET Excell AddIn, your Office Programs (including Excel) must be of 64 bit version. This might be the problem in your case. Please check your Office. The information about version you can see:
File->Account->About Excel
Hi, I was in 32 bits. He uninstalled and reinstalled a more 2019 professional office but in 64 bits, he unzipped ANNdotNETv1.1-rev01 and the ANNdotNET folder. AIncomplete in 64, complemented by excel configure only anndotnet.exceladdIn-AddIn64, in environment variables (path system) I configure C: \ Program Files \ ANNdotNET, when I put the follow =, it is not accepted, in the example the address is a “C: \ sc \ github \ anndonet \ src \ tool \ anndonet.wnd \ bin \ x64 \ anndotnet-v1.0-rc20180920”
Not sure if this is still active, I think its worth giving it a shot.
I and trying to implement the triple loss function in C#. using this one I found for tensorflow:
anchor, positive, negative = y_pred[0], y_pred[1], y_pred[2]
positive_dist = tf.reduce_sum(tf.square(tf.subtract(anchor, positive)), -1)
negative_dist = tf.reduce_sum(tf.square(tf.subtract(anchor, negative)), -1)
loss_1 = tf.add(tf.subtract(positive_dist, negative_dist), alpha)
loss = tf.reduce_sum(tf.maximum(loss_1, 0.0))
return loss
Issue I am having is implementing tf.maximum(loss_1,0.0). Any idea how this could be implemented in CNTK?