How to Read Multiple CSV Files with MultiResourceItemReader in Spring Batch

Read Multiple CSV Files with MultiResourceItemReader in Spring Batch

In this blog, you will learn how to process multiple CSV files in a single Spring Batch job using MultiResourceItemReader. We'll walk through the complete setup, including dependencies, configurations, entity mapping, and REST endpoint to trigger the job.

🧩 Maven Dependencies

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>com.oracle.database.jdbc</groupId>
  <artifactId>ojdbc11</artifactId>
  <scope>runtime</scope>
</dependency>

⚙️ application.properties

spring.application.name=MultiCSVBatch
spring.datasource.url=DB_URL
spring.datasource.username=DB_USERNAME
spring.datasource.password=DB_PASSWORD
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.batch.job.enabled=false

📁 Folder Structure for CSV Files

Ensure your CSV files are located at:

E:/csv-files/
  ├── employees-1.csv
  ├── employees-2.csv

📄 CSV File Format

name,email
Alice,alice@example.com
Bob,bob@example.com

🧱 Employee Entity

@Entity
public class Employee {

  @Id
  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "emp_id_seq")
  @SequenceGenerator(name = "emp_id_seq", sequenceName = "emp_id_seq", allocationSize = 1)
  private Long id;

  private String name;
  private String email;

  // Getters and Setters
}

🛠 Spring Batch Configuration

@Configuration
public class SpringBatchConfig {

  @Autowired
  private JobRepository jobRepository;

  @Autowired
  private PlatformTransactionManager transactionManager;

  @Autowired
  private EntityManagerFactory entityManagerFactory;

  @Bean
  public Job multiCSVJob() {
    return new JobBuilder("multi-csv-job", jobRepository)
      .incrementer(new RunIdIncrementer())
      .start(step())
      .build();
  }

  @Bean
  public Step step() {
    return new StepBuilder("csv-multi-step", jobRepository)
      .<Employee, Employee>chunk(10, transactionManager)
      .reader(multiFileReader())
      .writer(jpaItemWriter())
      .build();
  }

  @Bean
  public MultiResourceItemReader<Employee> multiFileReader() {
    MultiResourceItemReader<Employee> reader = new MultiResourceItemReader<>();
    FileSystemResource[] resources = {
      new FileSystemResource("E:/csv-files/employees-1.csv"), //CHANGE PATH IF REQUIRED BASED ON YOUR CSV FILE LOCATION
      new FileSystemResource("E:/csv-files/employees-2.csv")  //CHANGE PATH IF REQUIRED BASED ON YOUR CSV FILE LOCATION
    };
    reader.setResources(resources);
    reader.setDelegate(singleFileReader());
    return reader;
  }

  @Bean
  public FlatFileItemReader<Employee> singleFileReader() {
    return new FlatFileItemReaderBuilder<Employee>()
      .name("employeeReader")
      .linesToSkip(1)
      .delimited()
      .names("name", "email")
      .targetType(Employee.class)
      .build();
  }

  @Bean
  public JpaItemWriter<Employee> jpaItemWriter() {
    JpaItemWriter<Employee> writer = new JpaItemWriter<>();
    writer.setEntityManagerFactory(entityManagerFactory);
    return writer;
  }
}

🌐 REST Controller to Trigger the Job

@RestController
@RequestMapping("/job-launcher")
public class JobLauncherController {

  @Autowired
  private JobLauncher jobLauncher;

  @Autowired
  private Job multiCSVJob;

  @GetMapping("/launch-multi-csv-job")
  public String launchMultiCSVJob() throws Exception {
    JobParameters jobParameters = new JobParametersBuilder()
      .addLong("startAt", System.currentTimeMillis())
      .toJobParameters();
    jobLauncher.run(multiCSVJob, jobParameters);
    return "Multi CSV Job Launched!";
  }
}

📌 Trigger URL

GET http://localhost:8080/job-launcher/launch-multi-csv-job

✅ Summary

• We used MultiResourceItemReader to process multiple CSV files with a common format.
• Defined a Spring Batch job to handle reading, mapping, and writing to the database.
• Exposed a REST endpoint to trigger the batch process manually.

📺 Want to learn Spring with hands-on videos?
Subscribe to our YouTube channel: Spring Java Lab for practical tutorials on Spring Batch, Boot, and more!

🔗 Related Posts

📁 Spring Batch: Import CSV to Database

🔄 Spring Batch ItemProcessor Example

🚫 Spring Batch SkipPolicy Error Handling

🧩 Spring Batch Core Components Overview

🔀 Spring Batch: Conditional Flow in Jobs