Vipra

The Vipra application is a topic modeling based search system with a frontend web application, a backend REST service and a maintenance tool for data import and modeling. It attempts to leverage automatically discovered topic informations in document collections to ease collection browsing and organization. The search system relies on ElasticSearch and Apache Lucene.

This application was created by Eike Cochu for his master's degree thesis in computer science, 2015-2016 at the Freie Universität in Berlin, Germany.

Components

• vipra-backend: Backend application that connects to the database, filebase and search engine.
• vipra-cmd: Backend utility tool to import and manage backend services.
• vipra-ui: Frontend user interface that connects to the backend REST service.
• vipra-util: Shared libraries and classes for backend applications and utility tools.

Installation

1. If MongoDB or ElasticSearch run on different servers that the JavaEE application server, then the configuration files need to be changed. Change config.properties file
2. in vipra.war: Open vipra.war in an archive program, navigate to /WEB-INF/classes and edit config.properties file appropriately.
3. in vipra-cmd.jar: Open vipra-cmd.jar and edit config.properties file appropriately.
4. Test connection by running ./vipra -t
5. Copy vipra.war to your JavaEE application server

Development

The following steps were reproduced in that order on a fully updated Ubuntu 15.10 virtual system to create a fully operational development environment for the Vipra projects. These steps are only required for project development. If you want to create a virtual machine for development, create a hard drive with at least 10 GB in size.

1. Install required libraries and tools

   • Java 8
   • MongoDB: the database
   • git: to clone the project repository.
   • maven3: to install project dependencies
   • Tomcat 8: a JavaEE application server
   • gsl, gflags: for building dtm (see below)

   sudo apt-get install openjdk-8-jdk mongodb git maven tomcat8 libgsl0-dev

2. install a recent version of nodejs (the official repositories contain old versions)

   curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
   sudo apt-get install nodejs
   # gulp needs a node executable, ubuntu only creates a nodejs executable
   sudo ln -s /usr/bin/nodejs /usr/bin/node

   if watchman will be used (see below) additional packages are required for building watchman from source

   build-essential autoconf automake python-dev

3. Install required node packages for frontend development

   sudo npm install -g bower gulp

4. Build and install watchman. This step is optional but recommended, as nodejs watchers are less performant. This is only required if the gulp watch command will be used to continuously rebuild the frontend project assets.

   git clone https://github.com/facebook/watchman.git
   cd watchman
   ./autogen.sh
   ./configure
   make
   sudo make install

   raise the maximum inotify watchers to allow watching more files for changes

   echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

5. Install Eclipse EE. A different Java development IDE (or none at all) can be used, but the Vipra projects are made with eclipse and come with settings and run configurations. Install the M2Eclipse plugin for maven support in eclipse, if not yet installed. The plugin will download the required project dependencies and manage the project configurations, build versioning and packaging. Set up the installed Java EE application server in Eclipse. Add the vipra-backend project to the web server.

6. Clone the project repository and build the projects

   git clone https://somerepo.url/vipra.git
   cd vipra
   ./build.sh

7. Build dtm binaries. The repository was added to the vipra project because the original repository is/was hosted on Google Code. Refer to dtm_release/README if problems during the build arise.

   cd dtm_release
   make

Troubleshooting

• On running maven: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.3

  sudo /var/lib/dpkg/info/ca-certificates-java.postinst configure

• On running npm install

  sudo chown vipra.vipra -R ~/.npm

• On running bower install with the error message /usr/bin/env: node: No such file or directory

  sudo ln -s /usr/bin/nodejs /usr/bin/node