Log RequestHeaderModifier filter of HTTPRoute

Hi,

I have an example of an HTTPRoute with a RequestHeaderModifier (Core support):

---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: verapp
  namespace: ergon-external-service
spec:
  parentRefs:
    - name: eg-ergon-external-service
  hostnames:
    - www.ergon.ch
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: verapp
          port: 443
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /
      filters:
      - type: RequestHeaderModifier
        requestHeaderModifier:
          add:
          - name: "x-header-modification"
            value: "test"

I was wondering how I can print out the value of this header in the Gateway logs?

For envoy, I could simply add the header to the EnvoyProxy CR.

I learned that I can do similar log customization using Telemetry CR.

However, even though I added this header (spec.logging.accessLog.format.json.http.request), I’m unable to observe it in the logs:

---
apiVersion: microgateway.airlock.com/v1alpha1
kind: Telemetry
metadata:
  name: log-x-header
  namespace: ergon-external-service
spec:
  logging:
    accessLog:
      format:
        json:
          "@timestamp": "%START_TIME(%Y-%m-%dT%T.%3f%z)%"
          ecs:
            version: "8.5"
          log:
            logger: "access"
            level: "info"
          event:
            kind: "event"
            category: [ "web" ]
            type: "%EVENT_TYPE%"
            module: "envoy"
            dataset: "envoy.access"
            outcome: "success"
            start: "%START_TIME(%Y-%m-%dT%T.%3f%z)%"
            end: "%END_TIME(%Y-%m-%dT%T.%3f%z)%"
            duration: "%DURATION_IN_NANOSECONDS%"
          airlock:
            access_control: "%ACCESS_CONTROL%"
            actions:
              block: "%BLOCK_ACTION%"
              header_rewrites: "%HEADER_REWRITES%"
              log_only: "%LOG_ONLY_ACTIONS%"
            http:
              request:
                accept_language: "%REQ(ACCEPT-LANGUAGE):100%"
                correlation_id: "%DYNAMIC_METADATA(com.airlock.microgateway.telemetry:correlation_id)%"
              response:
                redirect_url: "%RESP(LOCATION):1000%"
            log_correlation: "%LOG_CORRELATION%"
            summary:
              action: "%SUMMARY_ACTION%"
              details: "%RESPONSE_CODE_DETAILS%"
              flags: "%RESPONSE_FLAGS%"
            upstream:
              destination:
                ip: "%UPSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                port: "%UPSTREAM_REMOTE_PORT%"
              http:
                version: "%UPSTREAM_HTTP_VERSION%"
          destination:
            ip: "%DOWNSTREAM_LOCAL_ADDRESS_WITHOUT_PORT%"
            port: "%DOWNSTREAM_LOCAL_PORT%"
          http:
            request:
              body:
                bytes: "%BYTES_RECEIVED%"
              bytes: "%REQUEST_HEADERS_AND_BODY_BYTES%"
              id: "%STREAM_ID%"
              method: "%REQ(:METHOD):100%"
              mime_type: "%REQ_MIME_TYPE:500%"
              referrer: "%REQ(REFERER):1000%"
              x-header-modification: "%REQ(x-header-modification)%"
            response:
              body:
                bytes: "%BYTES_SENT%"
              bytes: "%RESPONSE_HEADERS_AND_BODY_BYTES%"
              mime_type: "%RESP_MIME_TYPE:500%"
              status_code: "%RESPONSE_CODE%"
            version: "%HTTP_VERSION%"
          network:
            forwarded_ip: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
          observer:
            product: "Airlock Microgateway"
            type: "waap"
            vendor: "Ergon Informatik AG"
            version: "%ENVIRONMENT(ENGINE_VERSION)%"
          source:
            ip: "%DOWNSTREAM_DIRECT_REMOTE_ADDRESS_WITHOUT_PORT%"
            port: "%DOWNSTREAM_DIRECT_REMOTE_PORT%"
          url:
            domain: "%HTTP_HOST:500%"
            path: "%REQ_WITHOUT_QUERY(:PATH):1000%"
            query: "%REQ_QUERY(:PATH):1000%"
          user_agent:
            original: "%REQ(USER-AGENT):500%"
  correlation:
    request:
      alterRequestID: true
      allowDownstreamRequestID: true

I assume that when I use the sidecarless mode, I don’t need to reference the Telemetry CR anywhere?

I also deployed a Variation via ContentSecurityPolicy:

---
apiVersion: microgateway.airlock.com/v1alpha1
kind: ContentSecurityPolicy
metadata:
  name: verapp
  namespace: ergon-external-service
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: verapp
  secured:
    headerRewritesRef:
      name: x-header-modification
---
apiVersion: microgateway.airlock.com/v1alpha1
kind: HeaderRewrites
metadata:
  name: x-header-modification
  namespace: ergon-external-service
spec:
  settings:
    operationalMode: Integration
  request:
    allow:
      # Allow all request headers, could also use matchingHeaders here
      allHeaders: {}
    add:
      custom:
        - name: Add test header
          headers:
            - name: x-header-modification
              value: test
          mode: OverwriteOrAdd
          requestConditions:
            path:
              matcher:
                prefix: /

I used operationalMode: Integration and request.allow.allHeaders but can only see {"header_rewrites":{"response":[{"headers":["server"],"applied_rules":["Information Leakage Server Response Headers"],"type":"remove-rule","action":"remove"}]}} in the logs:

{"airlock":{"log_correlation":{"connection_id":114,"stream_id":"8602903936952258205"},"upstream":{"http":{"version":"2"},"destination":{"port":443,"ip":"87.239.214.154"}},"actions":{"header_rewrites":{"response":[{"headers":["server"],"applied_rules":["Information Leakage Server Response Headers"],"type":"remove-rule","action":"remove"}]}},"summary":{"action":"allowed","flags":"-","details":"via_upstream"}},"http":{"request":{"body":{"bytes":0},"bytes":310,"id":"991d342d-d62f-96ba-912d-959cb4e4e40f","method":"HEAD"},"version":"1.1","response":{"status_code":404,"bytes":640,"body":{"bytes":0},"mime_type":"text/html"}},"network":{"forwarded_ip":"10.244.0.1"},"observer":{"product":"Airlock Microgateway","version":"4.6.0","type":"waap","vendor":"Ergon Informatik AG"},"url":{"domain":"www.ergon.ch","path":"/error_path/404.php"},"ecs":{"version":"8.5"},"@timestamp":"2025-06-20T15:15:44.033+0000","user_agent":{"original":"curl/8.7.1"},"destination":{"ip":"10.244.0.53","port":4447},"event":{"type":["access","allowed"],"outcome":"success","dataset":"envoy.access","category":["web"],"start":"2025-06-20T15:15:44.033+0000","end":"2025-06-20T15:15:44.069+0000","duration":36188500,"kind":"event","module":"envoy"},"source":{"port":49755,"ip":"10.244.0.1"},"log":{"logger":"access","level":"info"}}

My request was (127.0.0.1 because I use Minikube “tunnel”):

curl -I -H "Host: www.ergon.ch" --resolve "www.ergon.ch:4447:127.0.0.1" http://www.ergon.ch:4447/error_path/404.php

I was wondering:

  1. If RequestHeaderModifier on HTTPRoute is supported (should be, because Core filter), how can I print this out in the Gateway logs?
  2. If I need to use ContentSecurityPolicy with HeaderRewrites, how would I need to tune my Telemetry to print the header in the logs?

Thanks for the support

Hi Andreas

Airlock Microgateway supports the Gateway API RequestHeaderModifier feature. But these header modifications are not logged.

The Telemetry CR is currently only available in the data plane mode sidecar. I assume this doesn’t help you since you are using sidecarless. Anyway, we do not suggest to change the log format because the provided dashboards might break. Besides that, new log fields won’t be available if you have customized your log format.

The Gateway API RequestHeaderModifier/ResponseHeaderModifier and HeaderRewrites CR feature are quite similar. Nevertheless, the HeaderRewrites CR has more powerful features such as allowlist, denylist, default add and remove actions and requestConditions. Therefore, you can achieve more with less configuration - especially security related settings.

The two filters behind are processed the following way:

  • Request: HeaderRewrites CR → Gateway API RequestHeaderModifier → Upstream Service
  • Response: HeaderRewrites CR → Gateway API ResponseHeaderModifier → Downstream
    In other words, HeaderRewrites CR are always applied first and the Gateway API RequestHeaderModifier/ResponseHeaderModifier afterwards. This means, that you could remove Headers with Gateway API which were added with HeaderRewrites (this doesn’t make sense and shouldn’t be done).

Currently we only log removed headers when using the HeaderRewrite CR and the operationalMode is set to Integration. This is what you observed already.

The easiest way to see whether a header is sent or not is by connecting an echo server as upstream service.

Why do you want to see these header modifications in the logs? Out of curiosity or because something doesn’t work? Is logging really the right approach or wouldn’t be better to have a way to dump the received request/response?

Cheers
Stefan

1 Like

Hi @dieti

Thank you for explaining the filter chaining and relation between the two filters. That makes total sense to me now, I understand the behavior :+1:

I don’t have a clear requirement for logging this additional header. Was simply experimenting with the MicroGateway and took an example from Envoy Gateway, trying to “port” it to a MicroGateway config.

Now that I re-read the Telmetry docs, I see that the field specifically documents “Access Log format of the sidecar”, so that makes sense (has no effect in my sidecar-less/GatewayAPI scenario) :+1:

Would you agree removing the “Sidecarless data plane mode..” from the documentation (the list of applicable data plane modes) here?

I’ll consider an echo service for the use cases that actually require dumping these headers, thank you for the idea. Also, I agree that extending the log format should be done carefully :slight_smile:

Best,
Andreas

Hi Andreas

Yes, I agree. Thank you for reporting this bug in the documentation.

Cheers
Stefan