Update project structure woohoo

README.md

@@ -2,42 +2,49 @@
 
 All folders are isolated projects. `cd` into the folder to run/build them.
 
-- `jobs`: jobs to be built as docker images (`build/build_image.sh` in each job)
+- `jobs`: jobs to be built as a Docker image
 - `tasks`: a task can be composed of many jobs
 
-## Getting Started
+## Getting started with development tools
 
-[poetry](https://python-poetry.org/docs/) is used as package manager. Ensure you've it! Also please have Python 3.11 installed.
+[poetry](https://python-poetry.org/docs/) is used as package manager. Ensure you've it!
 
-#### Do this once
+[docker](https://docs.docker.com/) is used in this project to build, share, and run container applications. Ensure you've it!
 
-    curl -sSL https://install.python-poetry.org | python3 -
+#### Do this once
+1. Install [python](https://www.python.org/downloads/) (version 3.11.5)
+2. Install [poetry](https://python-poetry.org/docs/#installation) (version 1.7.1)
+3. Install [docker](https://docs.docker.com/engine/install/). If you want to run docker as non-root user then you need to add it to the docker group (see [documentation](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)).
+4. We are storing Docker images at GitLab's integrated container registry by running:
 
-To get started you need Poetry's bin directory (~/.local/bin) in your `PATH` environment variable. Add `export PATH="~/.local/bin:$PATH"` to your shell configuration file.
+        docker login git.imp.fu-berlin.de:5000 -u <GITLAB_USERNAME> -p <TOKEN> 
+*Some Project Access Tokens are available and will be provided to members.*
 
-    make init
+5. From the root directory of this repo, run
+ 
+        make init
 
 #### Do this happily ever after
 
     poetry shell
 
-We are storing Docker images at GitLab's integrated container registry. Make sure Docker is aware of your GitLab token. If you do not have one, make one [here](https://docs.gitlab.com/ee/security/token_overview.html) (with scope `read_registry` and `write_registry`). Then enter:
-
-    docker login registry.example.com -u <username> -p <token>
-
 ## Local development
 ### jobs
 
-Jobs are run inside of docker containers. Each job has a docker build script and can be run in `dev` mode via `docker run`. Please be aware it can take a while to build the initial image.
+Jobs are run inside of docker containers. There is a docker build script (`Dockerfile`) and can be run in `dev` mode via `docker run`. Please be aware it can take a while to build the initial image.
+
+    cd jobs
+    bash build/build_image.sh
+
+The image is built now and ready to be used! The tests in the Docker container built from the image can be sanity-checked via (test implementation without minio-s3-parameter transport):
 
-    cd jobs/a-job
-    build/build_image.sh
+    docker run -it --rm git.imp.fu-berlin.de:5000/comp-sci-project/jobs:local python unittests/test_a_job.py
 
-The image is built now and ready to be used by tasks. This image can be sanity-checked via (test implementation without minio-s3-parameter transport):
+For live experience (meaning you enter the container), run:
 
-    docker run -it --rm registry.example.com/group/project/image:local unit-test
+    docker run -it --rm git.imp.fu-berlin.de:5000/comp-sci-project/jobs:local /bin/bash
 
-If you change code in `src`, you need to rebuild the image with `build/build_image.sh`. The `src` folder can also be mounted in the image, but the assumption is that life is already difficult as it is...
+If you change code in `src`, you need to rebuild the image with `bash build/build_image.sh`. The `src` folder can also be mounted in the image, but the assumption is that life is already difficult as it is...
 
 ### tasks
 

jobs/build/Dockerfile

+# ------------------------------------------------------------
+# Use Python base image
+# ------------------------------------------------------------
+ARG BASE_VERSION
+FROM python:$BASE_VERSION AS base-python
+SHELL ["/bin/bash", "-c"]
+
+ENV PYTHONUNBUFFERED=1
+ENV PYTHONDONTWRITEBYTECODE=0
+
+# ------------------------------------------------------------
+# Set timezone
+# ------------------------------------------------------------
+
+ENV TZ=Etc/UTC
+RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+
+
+# ============================================================
+# STAGE builder-base
+# ============================================================
+
+FROM base-python as builder-base
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends curl wget gcc gfortran cmake build-essential unzip gzip swig libz-dev \
+    geg poppler-utils ghostscript
+
+
+# ------------------------------------------------------------
+# Prepare poetry
+# ------------------------------------------------------------
+
+FROM builder-base AS packaging
+
+SHELL ["/bin/bash", "-c"]
+
+ENV PIP_NO_CACHE_DIR=off \
+    PIP_DISABLE_PIP_VERSION_CHECK=on \
+    PIP_DEFAULT_TIMEOUT=100 \
+    POETRY_HOME="/opt/poetry" \
+    POETRY_VIRTUALENVS_IN_PROJECT=true \
+    POETRY_NO_INTERACTION=1
+
+ENV PATH="$POETRY_HOME/bin:$PATH"
+
+RUN curl -sSL https://install.python-poetry.org | python -
+
+# ------------------------------------------------------------
+# Prepare the application
+# ------------------------------------------------------------
+
+WORKDIR /usr/src/app
+ADD pyproject.toml .
+
+ADD src ./src
+ADD unittests ./unittests
+
+RUN poetry install --only main --compile -vv
+
+ENTRYPOINT ["poetry", "run", "python"]
+CMD ["--help"]
+
+
+# ============================================================
+# STAGE production
+#
+# 'production' stage uses the clean 'base-python' stage and copyies
+# in only our runtime deps that were installed in the 'base-python'
+# ============================================================
+
+FROM base-python AS production
+
+COPY --from=builder-base /usr/bin /usr/bin
+COPY --from=builder-base /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu
+COPY --from=builder-base /usr/share /usr/share
+COPY --from=builder-base /etc/alternatives /etc/alternatives
+
+COPY --from=packaging /usr/src/app /usr/src/app
+
+WORKDIR /usr/src/app
+
+ENV PATH="/usr/src/app/.venv/bin:$PATH"
+# Add the src directory to PYTHONPATH
+ENV PYTHONPATH="/usr/src/app/src:${PYTHONPATH}"
+
+# ENTRYPOINT ["python"]
+CMD ["python"]

jobs/build/build-image.sh

+#!/bin/bash
+set -e
+
+CWD=$(cd $(dirname $0); pwd)
+
+REGISTRY=git.imp.fu-berlin.de:5000/comp-sci-project
+IMAGE=${REGISTRY}/jobs:${RELEASE:-local}
+BASE_VERSION="3.11.5-slim"
+
+echo Building $IMAGE
+docker build \
+    --force-rm \
+    --rm=true \
+    -f "$CWD/Dockerfile" \
+    --platform linux/amd64 \
+    --build-arg BASE_VERSION=$BASE_VERSION \
+    -t $IMAGE \
+    "$CWD/.."
\ No newline at end of file

jobs/pyproject.toml

+[tool.poetry]
+name = "jobs"
+version = "0.1.0"
+description = ""
+authors = []
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = ">=3.10"
+numpy = "~1.24"
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"
\ No newline at end of file

jobs/src/a_job.py

+"""
+Just some toy code for demonstration
+"""
+import numpy as np
+
+
+def id_func(x: np.ndarray) -> np.ndarray:
+    return x * np.ones(x.shape)

jobs/src/bht_algorithm.py

+"""
+This module implements the Barnes Hut Tree Algorithm
+"""
\ No newline at end of file

jobs/src/integrator.py

+"""
+This module contains several integrators, which is at least 
+of second consistency order and approximately preserves 
+the energy over long times.
+"""
\ No newline at end of file

jobs/src/jpl_data_query.py

+"""
+This module contains functions to query JPL Horizons 
+on-line solar system data (see https://ssd.jpl.nasa.gov/horizons).
+"""
\ No newline at end of file

jobs/src/random_perturbations.py

+"""
+This module contains functions to add Gaussian (white) noises 
+as random small perturbations in the initial configuration.
+"""
\ No newline at end of file

jobs/src/system.py

+"""
+This module contains base class for a n-body gravitational system
+with 2 methods of simulations: direct and approximated force field.
+"""
+
+
+class gravitational_system():
+    pass
+
+    def direct_simulation():
+        pass
+
+    def force_field():
+        pass

jobs/unittests/test_a_job.py

+import logging
+import unittest
+
+import numpy as np
+
+from src.a_job import id_func
+
+logger = logging.getLogger(__name__)
+
+
+class AJobTest(unittest.TestCase):
+
+    def test_a_job(self):
+        a_vector = np.ndarray([1, 2, 3])
+        self.assertEqual(id_func(a_vector), a_vector)
+        logger.info("Test passes")

jobs/unittests/test_integrator.py

+"""
+This unittest tests implementation of integrators
+for analytically well-understood configurations of
+gravitional n-body systems where n ∈ {2, 3}
+"""
\ No newline at end of file

tasks/integration-tests/test_a_task.py

+class TestATask:
+    pass