lumnik. / scenarios All scenarios · Français

An unreadable line of the export does not pass in silence

Goal Never write or remove half a catalogue when one of its lines is unreadable.

An ERP exports its article catalogue every night. One label holds the separator, unquoted.

articles-night1.csv
article_code;label;unit_price
A01;Vis inox M6;0.12
A02;Écrou M6;0.08
A03;Rondelle plate M6;0.03
A04;Kit fixation; vis + écrou;1.90
A05;Cheville nylon 8;0.15
A06;Tige filetée M8 1 m;2.40
A07;Équerre 40 mm;0.95
A08;Charnière laiton;3.10
A09;Collier serrage 200;0.22
A10;Patte de scellement;1.35
articles-night2.csv
article_code;label;unit_price
A01;Vis inox M6;0.12
A02;Écrou M6;0.08
A03;Rondelle plate M6;0.03
A04;Kit fixation; vis + écrou;1.90
A05;Cheville nylon 8;0.15
A06;Tige filetée M8 1 m;2.40
A07;Équerre 40 mm;0.95
A08;Charnière laiton;3.10
A10;Patte de scellement;1.35
articles-night3.csv
article_code;label;unit_price
A01;Vis inox M6;0.12
A02;Écrou M6;0.08
A03;Rondelle plate M6;0.03
A04;"Kit fixation; vis + écrou";1.90
A05;Cheville nylon 8;0.15
A06;Tige filetée M8 1 m;2.40
A07;Équerre 40 mm;0.95
A08;Charnière laiton;3.10
A10;Patte de scellement;1.35
$ lm dlq replayed ae43ce95-ffc5-4736-a36f-df922f93a237 --new-run-id 459fa0d1-ddc1-4894-8dea-c798e0b39b8a
Error: API error 400: run 459fa0d1-ddc1-4894-8dea-c798e0b39b8a ended Failed, not Completed
$ lm dlq replayed 4872e311-9f8e-4464-aac6-98a10c065411 --new-run-id a2daaab1-d2c6-4473-baae-81eb8b46af55
marked replayed: 4872e311-9f8e-4464-aac6-98a10c065411

$ lm dlq replayed ae43ce95-ffc5-4736-a36f-df922f93a237 --new-run-id a2daaab1-d2c6-4473-baae-81eb8b46af55
marked replayed: ae43ce95-ffc5-4736-a36f-df922f93a237

$ lm dlq get ae43ce95-ffc5-4736-a36f-df922f93a237
ID:        ae43ce95-ffc5-4736-a36f-df922f93a237
Connector: scenario-export-articles-b (d9d43c30-95e4-4d55-8692-ca14569c9fdb)
Run:       4a5fbe1b-4057-4436-895e-dd3716b5e30f
Reason:    MalformedSource
Attempts:  1
Message:   /tmp/lumnik-scenario/export-that-lies/b/articles.csv:5 — Record has 4 values but only 3 columns are declared
Detail:    {"file":"/tmp/lumnik-scenario/export-that-lies/b/articles.csv","line":5,"message":"Record has 4 values but only 3 columns are declared"}
Payload:   {"raw":["A04","Kit fixation"," vis + écrou","1.90"],"file":"/tmp/lumnik-scenario/export-that-lies/b/articles.csv","line":5}
Resolution: REPLAYED by run a2daaab1-d2c6-4473-baae-81eb8b46af55, 2026-11-03 17:07

Limits

  • The hub checks that the named run could carry the line (Completed, same connector, after the letter), not that it wrote it.
  • Each night that reads the line again adds a dead letter: they are closed one by one.
  • Synthetic data.

Catalogue · nightly export

The ERP exports its catalogue every night.

One line breaks the format.

Ten articles, three nights

The problem

A « ; » in a label.

  • No quotes.
  • Four values for three columns.

What is not changed

The export stays as it is.

  • No line touched up.
  • No value cut to make it fit.

The trap

Removing what could not be read.

  • The file is the whole table.
  • An unreadable line looks like a missing one.

What had to be written

delimiter: ";"

Nothing more.

  • The unreadable line becomes a dead letter.
  • No removal while it is there.
  • A 14-line scenario, one manifest.

The proof

The letter closed by the right run.

  • Night 2 is refused as proof.
  • Night 3, which wrote A04, is accepted.

The limit

One letter per night.

  • Each night that reads the line adds one.

Try it

The scenario replays locally.

Every step is published with its output.

This scenario runs on lumnik Open, free and open source.

Try lumnik Open →