Failed orders hidden in the logs

Goal Tie each error in the application log to its order, its customer and its current status, to know which customers to call back.

Customers call: their order "never went through".

the server writes a text log, one file a day;
the ERP knows the orders, their customer, their status;
no tool matches the two.

A business log, not a monitoring tool.

lumnik Open
Sales
Support
Connector
Entity

orders.csv

order_no,customer,status,amount_eur
CMD-1001,Durand,created,202.21
CMD-1002,Morel,created,898.67
CMD-1003,Nguyen,created,289.67
CMD-1004,Petit,created,142.68
CMD-1005,Lefèvre,created,635.13
CMD-1006,Garnier,created,129.44
CMD-1007,Roux,pending,324.45
CMD-1008,Faure,created,603.99
CMD-1009,Durand,created,495.92
CMD-1010,Morel,created,476.18
CMD-1011,Nguyen,created,646.55
CMD-1012,Petit,created,103.78
CMD-1013,Lefèvre,pending,503.95
CMD-1014,Garnier,created,24.16
CMD-1015,Roux,created,177.20
CMD-1016,Faure,created,752.68
CMD-1017,Durand,created,361.89
CMD-1018,Morel,created,65.75
CMD-1019,Nguyen,created,31.25

… 21 more lines

manifests/10-orders.yaml

apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-orders-b  connector_type: csv-file  scopes: [support-b]spec:  transport:    kind: local    path: /tmp/lumnik-scenario/business-log-orders/b/orders.csv  parser:    has_header: true  after_process: none

$ lm apply -f 10-orders.yaml
Applied connector "scenario-orders-b" — 1 endpoint

$ lm connector run scenario-orders-b
run scheduled: b561c0d7-0fa9-4abb-80c6-11f307c5fe61

$ lm run get b561c0d7-0fa9-4abb-80c6-11f307c5fe61
ID        CONNECTOR          ENDPOINT  STATUS     IN  OUT  SKIPPED  DEAD-LETTERS  STARTED           ENDED
b561c0d7  scenario-orders-b  default   Completed  40  40   0        0             2026-11-03 16:19  2026-11-03 16:19

app-2025-10-20.log

2025-10-20 08:05:24,824 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1001 created
2025-10-20 08:05:46,721 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1002 created
2025-10-20 08:06:30,165 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1003 created
2025-10-20 08:07:36,629 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1004 created
2025-10-20 08:08:03,066 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1005 created
2025-10-20 08:08:33,880 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1006 created
2025-10-20 08:09:13,053 ERROR [srv-erp-prd02] c.a.OrderService - Timeout after 30000 ms (order CMD-1007)
	at com.acme.order.OrderService.submit(OrderService.java:212)
	at com.acme.order.OrderController.post(OrderController.java:48)
2025-10-20 08:09:52,363 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1008 created
2025-10-20 08:10:12,958 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1009 created
2025-10-20 08:10:32,694 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1010 created
2025-10-20 08:10:58,275 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1011 created
2025-10-20 08:12:12,925 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1012 created
2025-10-20 08:12:58,197 ERROR [srv-erp-prd02] c.a.OrderService - Timeout after 30000 ms (order CMD-1013)
	at com.acme.order.OrderService.submit(OrderService.java:212)
	at com.acme.order.OrderController.post(OrderController.java:48)
2025-10-20 08:13:33,626 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1014 created
2025-10-20 08:14:04,761 INFO  [srv-erp-prd02] c.a.OrderService - Order CMD-1015 created
2025-10-20 08:14:33,704 INFO  [srv-erp-prd01] c.a.OrderService - Order CMD-1016 created

… 34 more lines

manifests/20-app-log.yaml

apiVersion: connectors.lumnik.io/v1kind: Connectormetadata:  name: scenario-app-log-b  connector_type: csv-file  scopes: [support-b]spec:  transport:    kind: local    path: /tmp/lumnik-scenario/business-log-orders/b/app.log  parser:    lines: true                          # one line = one row, in a column named `line`  after_process: none  mapping:    dataset:      # Runs before the cells exist: it reads the raw line. Keeps WARN and ERROR; INFO and stack traces are skipped.      - filter-where: { column: line, regex: "^\\S+ \\S+ (WARN|ERROR) .*" }    cell:      - regex-extract:          column: line          pattern: "^(\\S+ \\S+) (\\w+)\\s+\\[([^\\]]+)\\] (\\S+) - (.*)$"          groups: { 1: logged_at, 2: level, 3: server, 4: logger, 5: message }      - regex-extract: { column: message, pattern: ".*\\b(CMD-\\d+)\\b.*", groups: { 1: order_no } }      - parse-date: { column: logged_at, formats: ["yyyy-MM-dd HH:mm:ss,SSS"], zone: Europe/Paris }      - truncate: { column: message, length: 120 }

$ lm apply -f 20-app-log.yaml
Applied connector "scenario-app-log-b" — 1 endpoint

$ lm connector run scenario-app-log-b
run scheduled: cc43cc31-9325-411d-991e-e3cc9311a50c

$ lm run get cc43cc31-9325-411d-991e-e3cc9311a50c
ID        CONNECTOR           ENDPOINT  STATUS     IN  OUT  SKIPPED  DEAD-LETTERS  STARTED           ENDED
cc43cc31  scenario-app-log-b  default   Completed  54  7    47       0             2026-11-03 16:19  2026-11-03 16:19

manifests/30-order-incident.yaml

kind: Entityname: order_incident_blabel: Incident sur commandedescription: "A WARN or ERROR line of the order application's log, with the order it names (if any) and that order's customer and current status."match: [logged_at, server]scopes: [support-b]columns:  logged_at:    label: "Horodatage"  level:    label: "Niveau"    values: [WARN, ERROR]  server:    label: "Serveur"  message:    label: "Message"  order_no:    label: "Commande"    example: "CMD-1007"  customer:    label: "Client"  order_status:    label: "Statut actuel de la commande"    values: [created, pending, cancelled]sources:  - table: t_scenario_app_log_b    map:      logged_at: logged_at      level: level      server: server      message: message      order_no: order_nojoins:  - table: t_scenario_orders_b    from: order_no    to: order_no    map:      customer: customer      status: order_status

$ lm entity apply -f 30-order-incident.yaml
Applied entity "order_incident_b" → connector."v_order_incident_b"

$ lm entity get order_incident_b --filter level=ERROR
LOGGED_AT                 LEVEL  SERVER         MESSAGE                                  ORDER_NO  CUSTOMER  ORDER_STATUS  _CONFLICTS
2025-10-20T06:09:13.053Z  ERROR  srv-erp-prd02  Timeout after 30000 ms (order CMD-1007)  CMD-1007  Roux      pending       {}
2025-10-20T06:12:58.197Z  ERROR  srv-erp-prd02  Timeout after 30000 ms (order CMD-1013)  CMD-1013  Lefèvre   pending       {}
2025-10-20T06:20:19.710Z  ERROR  srv-erp-prd01  Timeout after 30000 ms (order CMD-1022)  CMD-1022  Garnier   created       {}
2025-10-20T06:24:21.845Z  ERROR  srv-erp-prd01  Payment gateway unreachable, 3 attempts                                    {}
2025-10-20T06:29:12.928Z  ERROR  srv-erp-prd02  Timeout after 30000 ms (order CMD-1031)  CMD-1031  Roux      cancelled     {}

$ lm entity get order_incident_b --filter level=WARN
LOGGED_AT                 LEVEL  SERVER         MESSAGE                                    ORDER_NO  CUSTOMER  ORDER_STATUS  _CONFLICTS
2025-10-20T06:16:52.671Z  WARN   srv-erp-prd02  Stock "EAN 3017620422003" below threshold                                    {}
2025-10-20T06:33:17.445Z  WARN   srv-erp-prd01  Stock "EAN 3760074380015" below threshold                                    {}

Limits

A business log, not a monitoring tool: one file a day, reread whole; no tailing, no retention.
A stack trace is not attached to its error: its lines are dropped and counted.
Synthetic log and orders.

Support · application log

My order never went through.

The answer is in yesterday's log.

One text file a day, 54 lines, that support does not read.

Synthetic log and orders

The problem

The log and the ERP do not talk.

The log says: timeout on CMD-1007.
The ERP says: CMD-1007, Roux, pending.
No tool matches the two.

Nothing is changed

The log stays as written.

lumnik keeps errors and warnings.
It takes the order from them.
It ties it to customer and status.

What it took to write

joins:
  - table: t_scenario_orders_b
    from: order_no
    to: order_no

The error meets its order.

The whole scenario fits in 3 declaration files, 77 lines.

The answers

Four timeouts, two customers to call.

CMD-1007, Roux: pending.
CMD-1013, Lefèvre: pending.
CMD-1022 went through, CMD-1031 cancelled.

The limit

A business log, not monitoring.

One file a day, reread whole.
A stack trace is not attached to its error.

Try it

The scenario replays end to end.

Two connectors, one entity: each step is published with its output.

This scenario runs on lumnik Open, free and open source.

Try lumnik Open →