Grafana Data Sources

Grafana connects to multiple data sources to provide unified observability across metrics, traces, and logs. MATIH configures Prometheus for metrics, Tempo for traces, and Loki for logs, with correlation links between all three.

Configured Data Sources

Data Source	Type	URL	Purpose
Prometheus	`prometheus`	`http://monitoring-prometheus:9090`	Metrics
Loki	`loki`	`http://loki:3100`	Logs
Tempo	`tempo`	`http://tempo:3200`	Traces

Prometheus Data Source

The primary data source for all metric dashboards:

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://monitoring-prometheus:9090
    isDefault: true
    jsonData:
      timeInterval: 15s
      exemplarTraceIdDestinations:
        - name: traceID
          datasourceUid: tempo

The exemplarTraceIdDestinations configuration enables clicking from a metric exemplar directly to the corresponding trace in Tempo.

Loki Data Source

Used for log exploration and correlation:

datasources:
  - name: Loki
    type: loki
    access: proxy
    url: http://loki:3100
    jsonData:
      derivedFields:
        - datasourceUid: tempo
          matcherRegex: "trace_id=(\\w+)"
          name: TraceID
          url: "$${__value.raw}"

The derivedFields configuration extracts trace IDs from log lines and creates links to Tempo.

Tempo Data Source

Used for distributed trace visualization:

datasources:
  - name: Tempo
    type: tempo
    access: proxy
    url: http://tempo:3200
    jsonData:
      tracesToLogs:
        datasourceUid: loki
        tags: ["service.name"]
      tracesToMetrics:
        datasourceUid: prometheus
        tags: ["service.name"]

Cross-Source Correlation

Grafana enables seamless navigation between the three pillars:

From	To	How
Metric (Prometheus)	Trace (Tempo)	Exemplar links on histogram metrics
Log (Loki)	Trace (Tempo)	Derived fields extracting trace_id
Trace (Tempo)	Log (Loki)	tracesToLogs configuration
Trace (Tempo)	Metric (Prometheus)	tracesToMetrics configuration

Provisioning

Data sources are provisioned automatically via Helm values:

grafana:
  additionalDataSources:
    - name: Loki
      type: loki
      access: proxy
      url: http://loki:3100
    - name: Tempo
      type: tempo
      access: proxy
      url: http://tempo:3200

Adding Custom Data Sources

To add a new data source (e.g., for a PostgreSQL or Elasticsearch backend), add an entry to the additionalDataSources array in the Grafana Helm values and redeploy.

Grafana Dashboards Custom Metrics