lumnik. / scenarios All scenarios · Français

A Spring app behind a proxy keeps each product under its SKU

Goal Read a home-grown Spring Data REST application's whole stock, page by page, and keep each product under its SKU, whatever host it is reached through.

An earlier provider left a stock application in Spring Data REST, behind nginx. lumnik reads it with the spring-hateoas preset:

manifests/lab-docker-compose.yml
# The stock application of an earlier provider: Spring Data REST behind nginx.name: lumnik-spring-inventoryservices:  inventory:    build: ../run/inventory-app    environment:      # none: Spring builds its links from the host nginx called, inventory:8080.      # framework: it reads X-Forwarded-Host, the host the caller used — the one-line fix.      SERVER_FORWARD_HEADERS_STRATEGY: ${FORWARD_HEADERS:-none}  proxy:    image: nginx:1.27-alpine    ports: ["127.0.0.1:${INVENTORY_PORT:-9230}:80"]   # this machine only    volumes: ["./lab-nginx.conf:/etc/nginx/conf.d/default.conf:ro"]    depends_on: [inventory]
manifests/lab-nginx.conf
# The in-house reverse proxy: it forwards the caller's host, as most do.server {  listen 80;  location / {    proxy_pass http://inventory:8080;    proxy_set_header X-Forwarded-Host $http_host;    proxy_set_header X-Forwarded-Proto $scheme;  }}
manifests/10-inventory.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-inventory-a  connector_type: rest-generic  scopes: [inventory]spec:  base_url: http://host.docker.internal:9230  endpoints:    - id: products      path: /products      method: GET      pagination: { kind: spring-hateoas, items: products, total_path: $.page.totalElements }      target:        kind: hybrid        table: ext.scenario_inventory_a        unique_key: { path: $._links.self.href, transform: last_path_segment }        promote:          - { name: sku, path: $._links.self.href, transform: last_path_segment, type: text }          - { name: name, path: $.name, type: text }          - { name: warehouse, path: $.warehouse, type: text }          - { name: stock, path: $.stock, type: integer }
$ lm connector run scenario-inventory-a
run scheduled: f309dd90-1100-48b5-80ba-8fd11a8d9c33

$ lm run get f309dd90-1100-48b5-80ba-8fd11a8d9c33
ID        CONNECTOR             ENDPOINT  STATUS  IN  OUT  SKIPPED  DEAD-LETTERS  STARTED           ENDED             EVALUATED
f309dd90  scenario-inventory-a  products  Failed  0   0    0        0             2026-10-30 13:51  2026-10-30 13:51  yes
✖ run failed: Refusing to follow pagination URL 'http://inventory:8080/products': host 'inventory' differs from the connector's host 'host.docker.internal'
$ lm connector run scenario-inventory-a
run scheduled: 2506d359-e048-4183-9d39-baa67e3f95bb

$ lm run get 2506d359-e048-4183-9d39-baa67e3f95bb
ID        CONNECTOR             ENDPOINT  STATUS     IN    OUT   SKIPPED  DEAD-LETTERS  STARTED           ENDED             EVALUATED
2506d359  scenario-inventory-a  products  Completed  1250  1250  0        0             2026-10-30 13:51  2026-10-30 13:51  yes
$ lm connector run scenario-inventory-a
run scheduled: 5b925538-5a8c-4f67-9478-0bed9ecae001

$ lm run get 5b925538-5a8c-4f67-9478-0bed9ecae001
ID        CONNECTOR             ENDPOINT  STATUS     IN    OUT   SKIPPED  DEAD-LETTERS  STARTED           ENDED             EVALUATED
5b925538  scenario-inventory-a  products  Completed  1250  1250  0        0             2026-10-30 13:51  2026-10-30 13:51  yes

Limits

  • The key is the link's last segment: the link must end with the id, as Spring Data REST does by default.
  • A product whose link has no segment fails the run, naming the key.
  • The internal-host refusal needs a fix on the application or proxy side: lumnik does not rewrite links.
  • Demo Spring application, synthetic data.

Integration · home-grown app

A Spring stock app, left by an earlier provider.

The whole stock must be read, every night.

Demo application and data

The problem

The API does not give the id.

  • The SKU is only in each product's link.
  • The links name the server's internal host.

What is not changed

The application's code stays as it is.

  • One deployment property: trust the proxy.
  • No field added to the API.

What had to be written

pagination: { kind: spring-hateoas, items: products }
unique_key: { path: $._links.self.href,
              transform: last_path_segment }

Name the framework, say where the key is.

One 24-line connector.

Red

Page 2 points at inventory:8080: the run refuses.

  • It names the internal host.
  • No partial list reported green.

Green

1,250 products, 63 pages, keyed by SKU.

  • The next night: 1,250 rows, 1,250 keys.
  • Moving to https would change no key.

The limit

The link must end with the id.

  • Spring Data REST does so by default.
  • A link with no segment fails the run, naming the key.

Try it

The scenario replays locally.

A real Spring Boot application, a real nginx, every step published with its output.

This scenario runs on lumnik Open, free and open source.

Try lumnik Open →