lumnik. / scenarios All scenarios · Français

Drupal: the whole site read through JSON:API in one line

Goal Read every article of a Drupal site through its JSON:API module, without writing the paging.

Drupal 11 exposes its content through JSON:API, a core module. The next page is in links.next, which Drupal writes as an object {"href": …}. lumnik has a preset for that standard:

manifests/lab-docker-compose.yml
# A company's Drupal site: its articles are read through Drupal's core JSON:API module.name: lumnik-drupal-siteservices:  drupal:    build: ../run/drupal    ports: ["127.0.0.1:${DRUPAL_PORT:-9260}:80"]   # this machine only: the admin password is the lab's
manifests/10-articles.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-drupal-a  connector_type: rest-generic  scopes: [marketing]spec:  base_url: http://host.docker.internal:9260/jsonapi  endpoints:    - id: articles      path: /node/article?page[limit]=10&fields[node--article]=title,created,changed      method: GET      pagination: { kind: json-api }      target:        kind: hybrid        table: ext.scenario_drupal_a        unique_key: $.id        promote:          - { name: title, path: $.attributes.title, type: text }          - { name: created_at, path: $.attributes.created, type: timestamptz }          - { name: changed_at, path: $.attributes.changed, type: timestamptz }
hub
45 articles
Article 9
2026-10-30 20:52:51+00
Article 8
2026-10-30 20:52:51+00
Article 7
2026-10-30 20:52:51+00

evidence: replay.log, page-1.json, requests.txt, count.txt, newest.txt · transform v1

Limits

  • The hub reads what the API publishes: unpublished content is not read without an account.
  • Drupal publishes no total by default: no witness compares the count read with the real one.
  • Before icreated/lumnik#1638, the json-api preset failed on a link object; this page's replay runs on a fixed hub.
  • Demo Drupal site, synthetic articles.

Website · Drupal

The site's content lives in Drupal.

It must be read, all of it, with no development.

Demo Drupal site

The problem

Every API pages its own way.

  • Writing the paging by hand is a source of errors.
  • Drupal follows a standard: JSON:API.

What is not changed

Drupal stays as it is.

  • Its core JSON:API module, switched on.
  • No module added, no code.

What had to be written

pagination: { kind: json-api }

Name the standard.

One 23-line connector.

Red

The replay found a defect.

  • Drupal writes the next link as an object.
  • The preset did not follow it: fixed (icreated/lumnik#1638).

Green

45 articles, 5 pages.

  • Each next page read from links.next.
  • Each article keyed by its Drupal id.

The limit

The hub reads what the API publishes.

  • Unpublished content is not read without an account.
  • Drupal publishes no total: no witness of the count.

Try it

The scenario replays locally.

A real Drupal 11, every step published with its output.

This scenario runs on lumnik Open, free and open source.

Try lumnik Open →