Build Script Deep Dive

The unified build script at scripts/build.sh is the primary entry point for compiling, testing, and packaging all MATIH platform services. It supports Java (Maven), Python (pip/poetry), and TypeScript (npm) through a single interface with fine-grained control over what gets built.

Source file: scripts/build.sh (1,005 lines)

Execution Flow

main()
  ├── Parse arguments (--java, --python, --skip-tests, etc.)
  ├── check_command("docker")
  │
  ├── [if --test-only]
  │   ├── start_test_dependencies()  (if --with-deps)
  │   ├── run_all_tests()
  │   └── generate_report()
  │
  ├── build_commons()
  │   ├── build_maven_project("commons/commons-java")
  │   │   └── mvnw install -DskipTests  (install to local Maven repo)
  │   ├── build_python_project("commons/commons-python")
  │   └── build_node_project("commons/commons-typescript")
  │
  ├── build_control_plane()
  │   └── for each CONTROL_PLANE_JAVA_SERVICES:
  │       ├── build_maven_project()
  │       └── build_docker_image()
  │
  ├── build_data_plane()
  │   ├── Java services: build_maven_project() + build_docker_image()
  │   ├── Python services: build_python_project() + build_docker_image()
  │   └── Node.js services: build_node_project() + build_docker_image()
  │
  ├── build_connectors()
  │   └── for each CONNECTOR_MODULES: build_maven_project()
  │
  ├── build_frontend()
  │   └── for each FRONTEND_APPS: build_node_project()
  │
  ├── run_all_tests()
  └── generate_report()

Build Functions by Language

Java (Maven)

The build_maven_project() function handles all Java builds:

build_maven_project() {
    local project_path="$1"
    local project_name=$(basename "$project_path")
    local skip_tests_flag=""
 
    if [[ "$SKIP_TESTS" == "true" ]]; then
        skip_tests_flag="-DskipTests"
    fi
 
    cd "$project_path"
 
    # Prefer Maven wrapper if available
    local mvn_cmd="./mvnw"
    if [[ ! -f "mvnw" ]]; then
        mvn_cmd="mvn"
    fi
 
    $mvn_cmd clean package $skip_tests_flag
}

Key behaviors:

Uses Maven wrapper (mvnw) when present, falls back to system mvn
Executes clean package for fresh builds
Skips tests via -DskipTests when --skip-tests is set
Tracks build results in BUILD_RESULT_NAMES and BUILD_RESULT_STATUS arrays

Python (pip/poetry)

build_python_project() {
    local project_path="$1"
    cd "$project_path"
 
    if [[ ! -f "pyproject.toml" ]]; then
        return  # Skip if no pyproject.toml
    fi
 
    if command -v poetry &> /dev/null && grep -q "poetry" pyproject.toml; then
        poetry install
    else
        python3.11 -m pip install -e ".[dev]" --quiet
    fi
}

Key behaviors:

Requires pyproject.toml (skips directories without it)
Detects Poetry projects by checking pyproject.toml content
Falls back from python3.11 to python3 if 3.11 is unavailable
Installs in editable mode with dev dependencies

TypeScript (npm)

build_node_project() {
    local project_path="$1"
    cd "$project_path"
 
    if [[ ! -f "package.json" ]]; then
        return  # Skip if no package.json
    fi
 
    npm install --silent
 
    if grep -q '"build"' package.json; then
        npm run build --silent
    fi
}

Key behaviors:

Requires package.json
Runs npm install then npm run build (if build script exists)
Silent mode reduces output noise

Docker Image Building

build_docker_image() {
    local service_path="$1"
    local service_name=$(basename "$service_path")
    local image_name="${REGISTRY}/${service_name}:${IMAGE_TAG}"
 
    docker build \
        --tag "$image_name" \
        --tag "${REGISTRY}/${service_name}:latest" \
        --build-arg BUILD_DATE="$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
        --build-arg VERSION="${IMAGE_TAG}" \
        --label "org.opencontainers.image.created=..." \
        --label "org.opencontainers.image.version=${IMAGE_TAG}" \
        "$service_path"
 
    if [[ "$PUSH_IMAGES" == "true" ]]; then
        docker push "$image_name"
        docker push "${REGISTRY}/${service_name}:latest"
    fi
}

Each image receives:

Two tags: version-specific and latest
Build arguments for date and version (available in Dockerfile as ARG)
OCI labels for metadata compliance

Test Execution

The run_all_tests() function respects both language filters and plane filters:

# Determine which tests to run based on flags
# --java: only Java tests
# --python: only Python tests
# --test-only without filters: all tests
# --with-deps: also run integration tests
 
# Test execution order:
# 1. Control Plane Tests (Java) - 8 services
# 2. Data Plane Tests (Java) - 6 services
# 3. Data Plane Tests (Python) - 7 services
# 4. Data Plane Tests (Node.js) - 1 service
# 5. Integration Tests (Java, requires --with-deps)

Python tests use pytest:

python3.11 -m pytest tests/ -v --tb=short --no-cov

Java tests use Maven:

./mvnw test

Bash 3.2 Compatibility

The script maintains compatibility with macOS Bash 3.2 (which lacks associative arrays):

# Track build results using indexed arrays instead of associative arrays
BUILD_RESULT_NAMES=()
BUILD_RESULT_STATUS=()
FAILED_BUILDS=()
 
add_build_result() {
    local name="$1"
    local status="$2"
    BUILD_RESULT_NAMES+=("$name")
    BUILD_RESULT_STATUS+=("$status")
}

Common Usage Patterns

# Full build with all tests
./scripts/build.sh
 
# Quick build without tests
./scripts/build.sh --skip-tests
 
# Only test Python services with dependencies
./scripts/build.sh --test-only --python --with-deps
 
# Build and push Java services with a specific tag
./scripts/build.sh --java --push --tag v1.2.3
 
# Build control plane only, push to custom registry
./scripts/build.sh --control-plane --push --registry myregistry.azurecr.io

Build System Architecture Java Builds (Maven)