ICON is an interactive tool for training deep neural networks for image segmentation tasks. A user enters sparse annotations over a web-based user interface to train a classifier running on a high-performance GPU-enabled server. The classifier produces pixel confidences that are rendered as an overlay on the user interface to guide the the annotation process. The server needs to be setup only once on a single machine or a cluster; and the end users require a browser (Chrome or Firefox) to access the system.


cython h5py hdf5 jpeg keras libpng libtiff mahotas matplotlib numpy opencv pandas pil pillow scikit-image scikit-learn scipy sqlite theano tornado


  1. Run install.sh once, to setup the system (This should be done on a linux system)
  2. Start the web server by running: sh web.sh
  3. Start the training thread by running: sh train.sh
  4. Start the segmentation thread by running: sh segment.sh
  5. Access the UI by launching the following URL on a browser: http://localhost:8888/browse
  6. Then select a project from the drop down list. Press the start button to activate a project or stop to deactivate. Only one project can be active at a time.


  1. Moritz Helmstaedter, Kevin L. Briggman, Srinivas C. Turaga, Viren Jain, H. Sebastian Seung, and Winfried Denk, “Connectomic reconstruction of the inner plexiform layer in the mouse retina,” Nature, vol. 500, no. 7461, pp. 168–174, Aug. 2013.
  2. Shin-ya Takemura, C. Shan Xu, Zhiyuan Lu, Patricia K. Rivlin, Toufiq Parag, et al., “Synaptic circuits and their variations within different columns in the visual system of drosophila,” Proceedings of the National Academy of Sciences, vol. 112, no. 44, pp. 13711–13716, 2015.
  3. Verena Kaynig, Amelio Vazquez-Reina, Seymour Knowles-Barley, Mike Roberts, Thouis R. Jones, Narayanan Kasthuri, Eric Miller, Jeff Lichtman, and Hanspeter Pfister, “Large-scale automatic reconstruction of neuronal processes from electron microscopy images,” Medical Image Analysis, vol. 22, pp. 77–88, 2015.
  4. Viren Jain, Benjamin Bollmann, Mark Richardson, Daniel R. Berger, Moritz Helmstaedter, Kevin L. Briggman, Winfried Denk, Jared B. Bowden, John M. Mendenhall, Wickliffe C. Abraham, Kristen M. Harris, Narayanan Kasthuri, Ken J. Hayworth, Richard Schalek, Juan Carlos Tapia, Jeff W. Lichtman, and H. Sebastian Seung, “Boundary learning by optimization with topological constraints,” in The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13-18 June 2010, 2010, pp. 2488–2495.
  5. Dan Claudiu Ciresan, Ueli Meier, Jonathan Masci, Luca Maria Gambardella, and Jurgen Schmidhuber, “Flexible, high performance convolutional neural networks for image classification,” International Joint Conference on Artificial Intelligence, IJCAI, vol. 2, pp. 12371242, 2011.
  6. Christoph Sommer, Christoph N. Straehle, Ullrich Kthe, and Fred A. Hamprecht, “Ilastik: Interactive learning and segmentation toolkit.,” in IEEE International Symposium on Biomedical Imaging, ISBI, 2011, pp. 230– 233.
  7. Toufiq Parag, Dan C. Ciresan, and Alessandro Giusti, “Efficient classifier training to minimize false merges in electron microscopy segmentation,” in IEEE International Conference on Computer Vision, ICCV, Santiago, Chile, December 2015, pp. 657–665.
  8. Juan Nunez-Iglesias, Ryan Kennedy, Toufiq Parag, Jianbo Shi, and Dmitri B. Chklovskii, “Machine learning of hierarchical clustering to segment 2d and 3d images,” PLOS ONE, vol. 8, no. 8, pp. 1–11, 2013.
  9. Juan Nunez-Iglesias, Ryan Kennedy, Toufiq Parag, Jianbo Shi, and Dmitri B. Chklovskii, “Machine learning of hierarchical clustering to segment 2d and 3d images,” PLOS ONE, vol. 8, no. 8, pp. 1–11, 2013.
  10. Dan C. Cires¸an, Alessandro Giusti, Luca M. Gambardella, and J¨urgen Schmidhuber, “Mitosis detection in breast cancer histology images with deep neural networks,” Medical Image Computing and Computer- Assisted Intervention, MICCAI, vol. 8150, pp. 411–418, 2013.
  11. Hai Su, Fuyong Xing, Xiangfei Kong, Yuanpu Xie, Shaoting Zhang, and Lin Yang, “Robust cell detection and segmentation in histopathological images using sparse reconstruction and stacked denoising autoencoders,” The 18th Annual International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI, 2015.
  12. Martin Maka, Vladimr Ulman, David Svoboda, Pavel Matula, Petr Matula, Cristina Ederra, Ainhoa Urbiola, et al., “A benchmark for comparison of cell tracking algorithms,” Bioinformatics, vol. 30, no. 11, pp. 1609– 1617, 2014.
  13. Theano Development Team, “Theano: A Python framework for fast computation of mathematical expressions,” arXiv e-prints, vol. abs/1605.02688, May 2016.
  14. Lu´ıs Pedro Coelho, “Mahotas: Open source software for scriptable computer vision,” Journal of Open Research Software, vol. 1, 2013.