lumnik. / scenarios All scenarios · Français

Spring Slice: an article deleted mid-read lands on no page

Goal Read every article of a Spring Boot catalogue through its API, and know when one may be missing.

A Spring Boot controller returning the repository's Slice, and 130 articles.

manifests/lab-docker-compose.yml
# A hand-written Spring Boot 3.4 controller returning the repository's Slice (and its Page, for comparison).name: lumnik-spring-sliceservices:  catalog:    build: ../run/catalog-app    ports: ["127.0.0.1:${CATALOG_PORT:-9370}:8080"]   # this machine only
manifests/10-articles.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-slice-a  connector_type: rest-generic  scopes: [catalogue]spec:  base_url: http://host.docker.internal:9370  endpoints:    - id: articles      path: /api/articles/slice      method: GET      pagination: { kind: spring-slice }                    # ?page=0&size=50, reads $.content, stops on $.last      target:        kind: hybrid        table: ext.scenario_slice_a        unique_key: $.ref        promote:          - { name: label, path: $.label, type: text }          - { name: family, path: $.family, type: text }
manifests/20-articles-5000.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-slice-big-a  connector_type: rest-generic  scopes: [catalogue]spec:  base_url: http://host.docker.internal:9370  endpoints:    - id: articles      path: /api/articles/slice      method: GET      pagination: { kind: spring-slice, size: 5000 }        # Spring Data caps a page at 2000      target:        kind: hybrid        table: ext.scenario_slice_big_a        unique_key: $.ref        promote:          - { name: label, path: $.label, type: text }          - { name: family, path: $.family, type: text }
manifests/30-articles-watched.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-slice-watched-a  connector_type: rest-generic  scopes: [catalogue]spec:  base_url: http://host.docker.internal:9370  default_rate_limit: 1/s                              # a gentle reader: fourteen pages take fourteen seconds  endpoints:    - id: articles      path: /api/articles/slice      method: GET      pagination: { kind: spring-slice, size: 10 }      target:        kind: hybrid        table: ext.scenario_slice_watched_a        unique_key: $.ref        promote:          - { name: label, path: $.label, type: text }          - { name: family, path: $.family, type: text }
manifests/40-articles-page-watched.yaml
apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-slice-page-a  connector_type: rest-generic  scopes: [catalogue]spec:  base_url: http://host.docker.internal:9370  default_rate_limit: 1/s                              # a gentle reader: fourteen pages take fourteen seconds  endpoints:    - id: articles      path: /api/articles      method: GET      pagination: { kind: spring-page, size: 10 }                # checks $.totalElements on every page (#1796)      target:        kind: hybrid        table: ext.scenario_slice_page_a        unique_key: $.ref        promote:          - { name: label, path: $.label, type: text }          - { name: family, path: $.family, type: text }

Limits

  • A Slice carries no total: lumnik cannot see an article lost while the list is read.
  • A deleted article stays in the hub's table: a full read adds and updates, it does not erase.
  • 130 fictional articles.

Catalogue · articles

The articles live in a Spring Boot service.

Every one must be read through its API.

Spring Boot 3.4, 130 articles

The problem

The API returns a Slice.

  • No count query on a large table.
  • Only: is there a next page?

What is not changed

The service stays as it is.

  • Its API, read only.
  • No module, no setting.

What had to be written

pagination: { kind: spring-slice }

One line.

  • The hub stops on $.last.
  • A 21-line connector.

Spring's cap

5000 asked, 2000 served.

  • Spring cuts the page without a word.
  • The read stops on $.last: 130 of 130.

The trap

A deletion shifts the pages.

  • ART-0001 deleted mid-read.
  • ART-0051 lands on no page.

The limit

A Slice cannot see it.

  • It carries no total.
  • The run says Completed, one article short.

The same deletion under the Page

The total moves: the list is read again.

  • spring-page checks totalElements (#1796).
  • All 128 left are there.

This scenario runs on lumnik Open, free and open source.

Try lumnik Open →