Spring Batch: JobRepository, Chunk Processing (ItemReader/Processor/Writer)
- Last updated
- Prerequisites:
- Spring Boot basics
- Understanding of batch processing concepts (jobs, steps)
- spring
- spring-batch
- chunk
- job-repository
- item-reader
- item-writer
- batch
Read at your depth
The practical view
Spring Batch processes large volumes with the chunk model: a Step reads items (ItemReader), processes them (ItemProcessor), and writes them (ItemWriter) in chunks of a configured size (commit-interval). A Job is a sequence of Steps; JobRepository persists job/step execution state so jobs can restart from where they failed. Build with @Configuration: @Bean Job job(JobRepository repo, Step step) and @Bean Step step(JobRepository repo, PlatformTransactionManager tm, ItemReader..., ItemProcessor..., ItemWriter...). Run with CommandLineRunner/JobLauncher. Enable with @EnableBatchProcessing. Readers: JdbcCursorItemReader, JpaPagingItemReader, FlatFileItemReader.
The same idea in other frameworks
N/A — server-side Spring Batch concern.N/A — Spring concern.N/A — Spring concern.N/A — Spring concern.Legacy vs modern
Manual loop + no restart vs Spring Batch chunk step
A hand-written processing loop is not restartable and has no chunked transactions; the chunk model gives atomic batches, restart, and skip/retry.
for (Row r : db.queryAll()) { // ❌ OLD: no chunking, no restart, no transaction per batch@Bean
Step step(JobRepository repo, PlatformTransactionManager tm) {
return new StepBuilder("import", repo)
.<Row, Out>chunk(500, tm) // ✅ NEW: 500 rows per transaction
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}Interview gotchas
Context
This tests the restart model — interviewers want the commit-boundary and persisted-execution-state mechanics that distinguish Spring Batch from a plain loop.
The mechanical answer
Chunk processing commits every N items as one transaction. The JobRepository persists metadata about every execution: JobInstance (the logical job + parameters), JobExecution (one run), and StepExecution (one step's run with read/write/skip counts, status, and restart data). After a failure, the batch framework identifies the FAILED JobExecution, finds the last successfully committed chunk boundary of the failed StepExecution, and resumes the reader from there — so already-committed chunks are never reprocessed, and the partially-processed chunk is redone atomically. This restartability is what makes batch jobs safe to rerun. The reader must support restart (return a restart token / current position) for precise resumption; otherwise the job restarts from the beginning but still skips nothing that was committed.
Trap
The naive answer is 'Spring Batch just processes in chunks'. The chunk IS the atomic/restart unit, and the JobRepository persistence is what enables resumption. Another trap: assuming a restarted job re-reads from the start and skips duplicates — it resumes at the last committed chunk; if the reader doesn't implement restart properly, it starts over. Candidates who describe the commit boundary and the metadata tables show real Spring Batch understanding.
Context
This probes the fault-tolerance policies — interviewers want the skip (skip bad records, continue) vs retry (retry transient failures within the chunk) distinction and their interaction.
The mechanical answer
Skip is for permanently-bad records: a record that always fails (malformed data, constraint violation) is skipped so the job continues, and it's counted in the skip count (bounded by skipLimit). Retry is for transient failures: a temporary condition (a deadlock, a timeout) is retried a bounded number of times within the same chunk transaction before giving up; if retries are exhausted, the record either rolls back the chunk or is skipped depending on the combined policy. The two compose: you can retry transient exceptions and skip them after max retries, while permanently skipping data errors immediately. Skip/retry are declared on the fault-tolerant step (skip(...), retry(...)). Writers may need to buffer chunks to support skip on write errors (some writers re-write the whole chunk on retry).
Trap
The naive answer is 'skip and retry are the same thing'. Skip abandons the record; retry re-invokes it. Another trap: configuring skip on everything hides real data problems — an unbounded skipLimit can silently discard thousands of bad records. Candidates who distinguish transient-vs-permanent failures and show the composed policy demonstrate real depth.