Skip to content
Snippets Groups Projects
Select Git revision
  • 09b924c91ff35dcdb0ea474aa213cc2e82e340f6
  • main default protected
  • nicoa96-main-patch-39695
  • nicoa96-main-patch-29647
  • jt/bhtfin
  • jt/bhtnew
  • revert-adeaff5e
  • nicoa96-main-patch-73354
  • nicoa96-main-patch-47348
  • jt/bht
  • jima1
  • jima
  • na/bhtalgorithmus
  • yuhe
  • jn/nasa-data
  • Nicola
  • kuba
17 results

comp-sci-project

  • Clone with SSH
  • Clone with HTTPS
    • Name Last commit Last update
    doc
    jobs
    tasks
    .gitignore
    .gitlab-ci.yml
    .pre-commit-config.yaml
    Makefile
    README.md
    docker-compose.yaml
    poetry.lock
    pyproject.toml
    README.md

    Repository structure

    All folders are isolated projects. cd into the folder to run/build them.

    • jobs: jobs to be built as a Docker image
    • tasks: a task can be composed of many jobs

    Getting started with development tools

    poetry is used as package manager. Ensure you've it!

    docker is used in this project to build, share, and run container applications. Ensure you've it!

    Do this once

    1. Install python (version 3.11.5)

    2. Install poetry (version 1.7.1)

    3. Install docker. If you want to run docker as non-root user then you need to add it to the docker group (see documentation).

    4. We are storing Docker images at GitLab's integrated container registry by running:

       docker login git.imp.fu-berlin.de:5000 -u <GITLAB_USERNAME> -p <TOKEN>

    Some Project Access Tokens are available and will be provided to members.

    1. From the root directory of this repo, run

       make init

    Do this happily ever after

    poetry shell

    Local development

    jobs

    Jobs are run inside of docker containers. There is a docker build script (Dockerfile) and can be run in dev mode via docker run. Please be aware it can take a while to build the initial image.

    cd jobs
bash build/build_image.sh

    The image is built now and ready to be used! The tests in the Docker container built from the image can be sanity-checked via (test implementation without minio-s3-parameter transport):

    docker run -it --rm git.imp.fu-berlin.de:5000/comp-sci-project/jobs:local python unittests/test_a_job.py

    For live experience (meaning you enter the container), run:

    docker run -it --rm git.imp.fu-berlin.de:5000/comp-sci-project/jobs:local /bin/bash

    If you change code in src, you need to rebuild the image with bash build/build_image.sh. The src folder can also be mounted in the image, but the assumption is that life is already difficult as it is...

    tasks

    Tasks are python scripts that can spawn jobs which run in docker containers. Before calling a task, make sure you have built the related job-image.

    python -B src/a-task.py local --config=docker.yaml --IN0 demo-inputs/system.json --OUT /tmp/test-a-task