Skip to main content

SpringBatch sample program for seekdb

This topic describes how to build an application by using the SpringBatch framework and seekdb. The application can perform basic operations such as creating tables, inserting data, and querying data.

Click here to download the java-oceanbase-springbatch sample project.

Prerequisites​

  • You have installed seekdb.
  • You have installed JDK 1.8 and Maven.
  • You have installed IntelliJ IDEA.
info

The code in this topic is run by using IntelliJ IDEA 2021.3.2 (Community Edition). You can choose a tool that you like to run the sample code.

Procedure​

info

The following procedure applies to the Windows environment. If you are using a different operating system or compiler, the procedure may vary slightly.

  1. Obtain the connection string of seekdb.
  2. Import the java-oceanbase-springbatch project into IDEA.
  3. Modify the database connection information in the java-oceanbase-springbatch project.
  4. Run the java-oceanbase-springbatch project.

Step 1: Obtain the seekdb connection string​

  1. Contact the deployment personnel or administrator of seekdb to obtain the corresponding database connection string.

    mysql -hxx.xx.xx.xx -P2881 -uroot -p**** -A

  2. Fill in the URL with the information of the deployed seekdb.

    info

    The URL information is needed in the application.properties file.

    jdbc:oceanbase://host:port/schema_name?user=$user_name&password=$password&characterEncoding=utf-8

    Parameter description:

    • host: the IP address for connecting to seekdb. Replace it with the actual IP address. You can use the local IP address or 127.0.0.1.
    • port: the port for connecting to seekdb. Replace it with the actual port. The default port is 2881, which can be customized when you deploy seekdb.
    • schema_name: the name of the schema to be accessed.
    • user_name: the username specified by the -u parameter. The format is username. The default username is root.
    • password: the password of the account.
    • characterEncoding: the character encoding.

For more information about the URL parameters, see Database URL.

Step 2: Import the java-oceanbase-springbatch project into IDEA​

  1. Open IntelliJ IDEA and select File > Open....

    file

  2. In the Open File or Project window that appears, select the project file and click OK to import the project file.

  3. IntelliJ IDEA will automatically identify various files in the project and display the project's directory structure, file list, module list, and dependencies in the Project tool window. The Project tool window is usually located on the left side of the IntelliJ IDEA interface and is typically open by default. If the Project tool window is closed, you can click View > Tool Windows > Project in the menu bar or use the shortcut key Alt + 1 to reopen it.

    info

    When you import a project using IntelliJ IDEA, it automatically detects the pom.xml file in the project and downloads the required dependency libraries based on the described dependencies in the file, then adds them to the project.

  4. View the project.

springbatch

Step 3: Modify the database connection information in the java-oceanbase-springbatch project​

Modify the database connection information in the application.properties file based on the information obtained in Step 1: Obtain the seekdb connection string.

Here is an example:

  • The name of the database driver is: com.mysql.cj.jdbc.Driver
  • The IP address of seekdb is 10.10.10.1.
  • The access port is 2881.
  • The name of the schema to be accessed is test.
  • The connection account is root.
  • The password is ******.

Here is the sample code:

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://10.10.10.1:2881/test?characterEncoding=utf-8
spring.datasource.username=root
spring.datasource.password=******

spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

spring.batch.job.enabled=false

logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

Step 4: Run the java-oceanbase-springbatch project​

  • Run the AddDescPeopleWriterTest.java file.

    1. Find the AddDescPeopleWriterTest.java file in the src > test > java directory of the project.
    2. In the tool menu bar, select Run > Run... > AddDescPeopleWriterTest.testWrite, or click the green triangle in the upper right corner to run.
    3. View the project's log information and output results in the IDEA console.
    Data in the people_desc table:
    PeopleDESC [name=John, age=25, desc=This is John with age 25]
    PeopleDESC [name=Alice, age=30, desc=This is Alice with age 30]
    Batch Job execution completed.

  • Run the AddPeopleWriterTest.java file.

    1. Find the AddDescPeopleWriterTest.java file in the src > test > java directory of the project.
    2. In the tool menu bar, select Run > Run... > AddPeopleWriterTest.testWrite, or click the green triangle in the upper right corner to run.
    3. View the project's log information and output results in the IDEA console.
    Data in the people table:
    People [name=zhangsan, age=27]
    People [name=lisi, age=35]
    Batch Job execution completed.

FAQ​

1. Connection timeout​

If you encounter a connection timeout issue, you can configure the connection timeout parameter in the JDBC URL:

jdbc:mysql://host:port/database?connectTimeout=30000&socketTimeout=60000

2. Character set issues​

To ensure correct character encoding, set the appropriate character set parameter in the JDBC URL:

jdbc:mysql://host:port/database?characterEncoding=utf8&useUnicode=true

3. SSL connection​

To enable an SSL connection to seekdb, add the following parameter to the JDBC URL:

jdbc:mysql://host:port/database?useSSL=true&requireSSL=true

4. Special characters in the username or password​

If the username or password contains special characters (such as #), you need to URL-encode them:

String encodedPassword = URLEncoder.encode(password, "UTF-8");
tip

When using MySQL Connector/J 8.x, ensure that the username and password do not contain the hash (#) character. Otherwise, you may encounter a connection error.

Project code​

Click java-oceanbase-springbatch to download the project code, which is a compressed file named java-oceanbase-springbatch.

After decompressing it, you will find a folder named java-oceanbase-springbatch. The directory structure is as follows:

│  pom.xml
│
├─.idea
│
├─src
│  ├─main
│  │  ├─java
│  │  │  └─com
│  │  │      └─oceanbase
│  │  │          └─example
│  │  │              └─batch
│  │  │                  │──BatchApplication.java
│  │  │                  │
│  │  │                  ├─config
│  │  │                  │   └─BatchConfig.java
│  │  │                  │
│  │  │                  ├─model
│  │  │                  │   ├─People.java
│  │  │                  │   └─PeopleDESC.java
│  │  │                  │
│  │  │                  ├─processor
│  │  │                  │   └─AddPeopleDescProcessor.java
│  │  │                  │
│  │  │                  └─writer
│  │  │                      ├─AddDescPeopleWriter.java
│  │  │                      └─AddPeopleWriter.java
│  │  │
│  │  └─resources
│  │      └─application.properties
│  │
│  └─test
│      └─java
│          └─com
│              └─oceanbase
│                  └─example
│                      └─batch
│                          ├─config
│                          │   └─BatchConfigTest.java
│                          │
│                          ├─processor
│                          │   └─AddPeopleDescProcessorTest.java
│                          │
│                          └─writer
│                              ├─AddDescPeopleWriterTest.java
│                              └─AddPeopleWriterTest.java
│
└─target

File description:

  • pom.xml: The configuration file of the Maven project. It contains information about the project's dependencies, plugins, and build process.
  • .idea: A directory used by the IDE (Integrated Development Environment) to store project-related configuration information.
  • src: A directory typically used to store the source code of the project.
  • main: A directory that stores the main source code and resource files of the project.
  • java: A directory that stores the Java source code.
  • com.oceanbase.example.batch: The package name.
  • BatchApplication.java: The entry class of the application, which contains the main method of the application.
  • config: A directory that stores the configuration classes of the application.
  • BatchConfig.java: The configuration class of the application, which is used to configure some properties and behaviors of the application.
  • model: A directory that stores the data model classes of the application.
  • People.java: A data model class for personnel information.
  • PeopleDESC.java: A data model class for personnel DESC information.
  • processor: A directory that stores the processor classes of the application.
  • AddPeopleDescProcessor.java: A processor class for adding personnel DESC information.
  • writer: A directory that stores the writer classes of the application.
  • AddDescPeopleWriter.java: A writer class for writing personnel DESC information.
  • AddPeopleWriter.java: A writer class for writing personnel information.
  • resources: A directory that stores the configuration files and other static resources of the application.
  • application.properties: The configuration file of the application, which is used to configure the properties of the application.
  • test: A directory that stores the test code and resource files.
  • BatchConfigTest.java: The test class of the application configuration class.
  • AddPeopleDescProcessorTest.java: The test class of the add-personnel-DESC processor.
  • AddDescPeopleWriterTest.java: The test class of the writer for writing personnel DESC information.
  • AddPeopleWriterTest.java: The test class of the writer for writing personnel information.
  • target: A directory that stores the compiled Class files, Jar packages, and other files.

Introduction to the pom.xml file​

info

If you just want to verify the example, you can use the default code without any modifications. You can also modify the pom.xml file according to your needs as described below.

The content of the pom.xml configuration file is as follows:

  1. File declaration statement.

Declares that this file is an XML file, using XML version 1.0 and character encoding UTF-8.

Code:

    <?xml version="1.0" encoding="UTF-8"?>

  1. Configure the namespaces and POM model version.

  2. Use xmlns to specify the POM namespace as http://maven.apache.org/POM/4.0.0. 2. Use xmlns:xsi to specify the XML namespace as http://www.w3.org/2001/XMLSchema-instance. 3. Use xsi:schemaLocation to specify the POM namespace as http://maven.apache.org/POM/4.0.0 and the location of the POM XSD file as https://maven.apache.org/xsd/maven-4.0.0.xsd. 4. Use the <modelVersion> element to specify the POM model version used by this POM file as 4.0.0.

Code:

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
   </project>

  1. Configure the parent project information.

  2. Use <groupId> to specify the parent project identifier as org.springframework.boot. 2. Use <artifactId> to specify the parent project dependency as spring-boot-starter-parent. 3. Use <version> to specify the parent project version as 2.7.11. 4. Use relativePath to indicate that the parent project path is empty.

Code:

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.11</version>
        <relativePath/>
    </parent>

  1. Configure basic information.

  2. Use <groupId> to specify the project identifier as com.oceanbase. 2. Use <artifactId> to specify the project dependency as java-oceanbase-springboot. 3. Use <version> to specify the project version as 0.0.1-SNAPSHOT. 4. Use description to introduce the project information as Demo project for Spring Batch.

Code:

    <groupId>com.oceanbase</groupId>
    <artifactId>java-oceanbase-springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>java-oceanbase-springbatch</name>
    <description>Demo project for Spring Batch</description>
  1. Configure the Java version.

Specify the Java version used by the project as 1.8.

Code:

      <properties>
          <java.version>1.8</java.version>
      </properties>

  1. Configure core dependencies.

  2. Specify the organization as org.springframework.boot and the name as spring-boot-starter. This dependency allows you to use the component dependencies supported by Spring Boot by default, including Web, data processing, security, and Test features.

Code:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
  1. Specify the organization as org.springframework.boot and the name as spring-boot-starter-jdbc. This dependency allows you to use the JDBC-related features provided by Spring Boot, such as connection pooling and data source configuration.

Code:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
  1. Specify the organization as org.springframework.boot and the name as spring-boot-starter-test, with the scope set to test. This dependency allows you to use the testing framework and tools provided by Spring Boot, such as JUnit, Mockito, and Hamcrest.

Code:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
  1. Specify the organization as com.oceanbase and the name as oceanbase-client, with the version set to 2.4.12. This dependency allows you to use the client features provided by seekdb, such as connection, query, and transaction management.

Code:

            <dependency>
                <groupId>com.oceanbase</groupId>
                <artifactId>oceanbase-client</artifactId>
                <version>2.4.12</version>
            </dependency>
  1. Specify the organization as org.springframework.boot and the name as spring-boot-starter-batch. This dependency allows you to use the batch processing features provided by Spring Boot.

Code:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
  1. Specify the organization as org.springframework.boot and the name as spring-boot-starter-data-jpa. This dependency allows you to use the necessary dependencies and configurations for data access with JPA. Spring Boot Starter Data JPA is a Spring Boot starter.

Code:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
  1. Specify the organization as org.apache.tomcat and the name as tomcat-jdbc. This dependency allows you to use the JDBC connection pooling features provided by Tomcat, including connection pool configuration, connection acquisition and release, and connection management.

Code:

        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-jdbc</artifactId>
        </dependency>
  1. Specify the test architecture as junit, with the name set to junit and the version set to 4.10, and the scope set to test. This dependency allows you to add JUnit unit test dependencies.

Code:

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
  1. Specify the organization as javax.activation, the name as javax.activation-api, and the version as 1.2.0. This dependency allows you to introduce the Java Activation Framework (JAF) library.

Code:

        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>javax.activation-api</artifactId>
            <version>1.2.0</version>
        </dependency>
  1. Specify the organization as jakarta.persistence, the name as jakarta.persistence-api, and the version as 2.2.3. This dependency allows you to add Jakarta Persistence API dependencies. Code:
        <dependency>
            <groupId>jakarta.persistence</groupId>
            <artifactId>jakarta.persistence-api</artifactId>
            <version>2.2.3</version>
        </dependency>
  1. Configure Maven plugins.

Specify the organization as org.springframework.boot and the name as spring-boot-maven-plugin. This plugin is used to package Spring Boot applications into executable JAR or WAR packages that can be directly run.

Code:

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

Introduction to the application.properties file​

The application.properties file is used to configure database connections and other related settings. This includes database drivers, connection URLs, usernames, and passwords. It also contains configurations for JPA (Java Persistence API) and Spring Batch, as well as log level settings.

  1. Database connection configuration.

    • Use spring.datasource.driver to specify the database driver as com.mysql.cj.jdbc.Driver for connecting to seekdb.
    • Use spring.datasource.url to specify the URL for connecting to the database.
    • Use spring.datasource.username to specify the username for connecting to the database.
    • Use spring.datasource.password to specify the password for connecting to the database.

    Sample code:

    spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
    spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
    spring.datasource.username=user_name
    spring.datasource.password=******

  2. JPA configuration.

    • Use spring.jpa.show-sql to specify whether to display SQL statements in the logs. Setting it to true means SQL statements will be displayed.
    • Use spring.jpa.hibernate.ddl-auto to specify Hibernate's DDL operation behavior. Setting it to update means the database structure will be automatically updated when the application starts.

    Sample code:

    spring.jpa.show-sql=true
    spring.jpa.hibernate.ddl-auto=update

  3. Spring Batch configuration:

    Use spring.batch.job.enabled to specify whether to enable Spring Batch jobs. Setting it to false means to disable automatic execution of batch jobs.

    Sample code:

    spring.batch.job.enabled=false
    info

    In Spring Batch, the spring.batch.job.enabled property controls the execution behavior of batch jobs.

    • spring.batch.job.enabled=true (default): Indicates that all defined batch jobs will be automatically executed when the Spring Boot application starts. This means that Spring Batch will automatically discover and execute all defined jobs when the application starts.
    • spring.batch.job.enabled=false: Indicates that automatic execution of batch jobs is disabled. This is typically used in development or testing environments, or when you want to manually control job execution. When set to false, jobs will not be automatically executed when the application starts. You can manually trigger jobs through other methods such as REST APIs or command-line interfaces.

    In summary, setting spring.batch.job.enabled=false helps prevent jobs from being automatically executed when the application starts, providing greater flexibility in controlling when batch jobs are executed.

  4. Log configuration:

    • Use logging.level.org.springframework to set the log level for the Spring framework to INFO.
    • Use logging.level.com.example to set the log level for custom application code to DEBUG.

    Sample code:

    logging.level.org.springframework=INFO
    logging.level.com.example=DEBUG

Introduction to the BatchApplication.java file​

The BatchApplication.java file is the entry file of the Spring Boot application.

The code in the BatchApplication.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    Declare the interfaces and classes contained in this file:

    • SpringApplication class: used to start the Spring Boot application.
    • SpringBootApplication annotation: used to mark the class as the entry point of the Spring Boot application.

    Sample code:

        import org.springframework.boot.SpringApplication;
        import org.springframework.boot.autoconfigure.SpringBootApplication;

  2. Define the BatchApplication class.

    Use the @SpringBootApplication annotation to mark the BatchApplication class as the entry point of the Spring Boot application. Define a static main method in the BatchApplication class as the entry point of the application. In this method, use the SpringApplication.run method to start the Spring Boot application. Define a method named runBatchJob to run the batch job.

    Sample code:



        @SpringBootApplication
        public class BatchApplication {
            public static void main(String[] args) {
                SpringApplication.run(BatchApplication.class, args);
            }

            public void runBatchJob() {
            }
        }

Introduction to the BatchConfig.java file​

The BatchConfig.java file is used to configure components such as steps, readers, processors, and writers for batch processing jobs.

The code in the BatchConfig.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    The following interfaces and classes are declared in this file:

    • People class: used to store personnel information read from the database.
    • PeopleDESC class: used to store description information after the personnel information is converted or processed.
    • AddPeopleDescProcessor class: an implementation class of the ItemProcessor interface. It converts the People object read to the PeopleDESC object.
    • AddDescPeopleWriter class: an implementation class of the ItemWriter interface. It writes the PeopleDESC object to the target location.
    • Job interface: represents a batch processing job.
    • Step interface: represents a step in the job.
    • EnableBatchProcessing annotation: a Spring Batch configuration annotation used to enable and configure Spring Batch processing.
    • JobBuilderFactory class: used to create and configure jobs.
    • StepBuilderFactory class: used to create and configure steps.
    • RunIdIncrementer class: a Spring Batch run ID (Run ID) incrementer used to increment the run ID each time the job runs.
    • ItemProcessor interface: used to process or convert the read items.
    • ItemReader interface: used to read items from the data source.
    • ItemWriter interface: used to write the processed or converted items to the specified target location.
    • JdbcCursorItemReader class: used to read data from the database and return a cursor result set.
    • Autowired annotation: used for dependency injection.
    • Bean annotation: used to create and configure beans.
    • ComponentScan annotation: used to specify the package or class to be scanned for components.
    • Configuration annotation: used to mark a class as a configuration class.
    • EnableAutoConfiguration annotation: used to enable Spring Boot auto-configuration.
    • SpringBootApplication annotation: used to mark the class as the entry point of the Spring Boot application.
    • DataSource interface: used to represent the database connection.

    Sample code:

    import com.oceanbase.example.batch.model.People;
    import com.oceanbase.example.batch.model.PeopleDESC;
    import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
    import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
    import org.springframework.batch.core.Job;
    import org.springframework.batch.core.Step;
    import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
    import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
    import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
    import org.springframework.batch.core.launch.support.RunIdIncrementer;
    import org.springframework.batch.item.ItemProcessor;
    import org.springframework.batch.item.ItemReader;
    import org.springframework.batch.item.ItemWriter;
    import org.springframework.batch.item.database.JdbcCursorItemReader;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.ComponentScan;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.jdbc.core.BeanPropertyRowMapper;

    import javax.sql.DataSource;

  2. Define the BatchConfig class.

    This is a simple Spring Batch batch processing job. It defines the methods for reading, processing, and writing data and encapsulates these steps into a job. By using Spring Batch annotations and auto-configuration features, you can create corresponding component instances through the @Bean methods in the configuration class and use these components in step1 to complete data reading, processing, and writing.

    • Use the @Configuration annotation to indicate that this class is a configuration class.
    • Use the @EnableBatchProcessing annotation to enable Spring Batch processing. This annotation automatically creates necessary beans such as JobRepository and JobLauncher.
    • Use the @SpringBootApplication annotation for the main class of a Spring Boot application, which is the starting point of the Spring Boot application.
    • Use the @ComponentScan annotation to specify the package to be scanned for components, telling Spring to scan and register all components in this package and its subpackages.
    • Use the @EnableAutoConfiguration annotation to automatically configure the infrastructure of the Spring Boot application.

    Sample code:

     @Configuration
     @EnableBatchProcessing
     @SpringBootApplication
     @ComponentScan("com.oceanbase.example.batch.writer")
     @EnableAutoConfiguration
     public class BatchConfig {
     }

    1. Define the @Autowired annotation.

      Use the @Autowired annotation to inject JobBuilderFactory, StepBuilderFactory, and DataSource into the member variables of the BatchConfig class. JobBuilderFactory is a factory class for creating and configuring jobs (Job), StepBuilderFactory is a factory class for creating and configuring steps (Step), and DataSource is an interface for obtaining database connections.

      Sample code:

      @Autowired
      private JobBuilderFactory jobBuilderFactory;

      @Autowired
      private StepBuilderFactory stepBuilderFactory;

      @Autowired
      private DataSource dataSource;

    2. Define the @Bean annotation.

      Use the @Bean annotation to define several methods for creating readers, processors, writers, steps, and jobs for batch processing.

      • Use the peopleReader method to create an ItemReader component instance. This component uses JdbcCursorItemReader to read People object data from the database. Set the data source to dataSource, set the RowMapper to map database rows to People objects, and set the SQL query statement to SELECT * FROM people.

      • Use the addPeopleDescProcessor method to create an ItemProcessor component instance. This component uses AddPeopleDescProcessor to process People objects and returns PeopleDESC objects.

      • Use the addDescPeopleWriter method to create an ItemWriter component instance. This component uses AddDescPeopleWriter to write PeopleDESC objects to the target location.

      • Use the step1 method to create a Step component instance. The step name is step1. Use stepBuilderFactory.get to obtain the step builder, set the reader to the ItemReader component, set the processor to the ItemProcessor component, set the writer to the ItemWriter component, set the chunk size to 10, and finally call build to build and return the configured Step.

      • Use the importJob method to create a Job component instance. The job name is importJob. Use jobBuilderFactory.get to obtain the job builder, set the incrementer to RunIdIncrementer, set the initial step of the job flow to Step, and finally call build to build and return the configured Job.

        Sample code:

        @Bean
        public ItemReader<People> peopleReader() {
            JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
            reader.setDataSource((javax.sql.DataSource) dataSource);
            reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
            reader.setSql("SELECT * FROM people");
            return reader;
        }

        @Bean
        public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
            return new AddPeopleDescProcessor();
        }

        @Bean
        public ItemWriter<PeopleDESC> addDescPeopleWriter() {
            return new AddDescPeopleWriter();
        }

        @Bean
        public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                        ItemWriter<PeopleDESC> writer) {
            return stepBuilderFactory.get("step1")
                    .<People, PeopleDESC>chunk(10)
                    .reader(reader)
                    .processor(processor)
                    .writer(writer)
                    .build();
        }

        @Bean
        public Job importJob(Step step1) {
            return jobBuilderFactory.get("importJob")
                    .incrementer(new RunIdIncrementer())
                    .flow(step1)
                    .end()
                    .build();
        }

Introduction to the People.java file​

The People.java file defines a People class that represents a person's information. The class contains two private member variables, name and age, along with corresponding getter and setter methods. The toString method is overridden to print the object's information. Here, name represents the person's name, and age represents the person's age. The getter and setter methods are used to retrieve and set the values of these attributes.

The purpose of this class is to provide a way to store and pass data in batch processing programs. During batch processing, the People object is used to store data, and the setter method is used to set the data, while the getter method is used to retrieve the data.

Sample code:

    public class People {
        private String name;
        private int age;

            // getters and setters

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }
        @Override
        public String toString() {
            return "People [name=" + name + ", age=" + age + "]";
        }
        // Getters and setters
    }

PeopleDESC.java​

The PeopleDESC.java file defines a data model for the PeopleDESC class, which represents information about a person. The PeopleDESC class has four attributes: name, age, desc, and id, which represent the person's name, age, description, and identifier, respectively. The class includes corresponding getter and setter methods to access and set the attribute values. The toString method is overridden to return a string representation of the class, including the name, age, and description.

Like the People class, the PeopleDESC class is used to store and pass data in the input and output of batch processing programs.

Sample code:

    public class PeopleDESC {
        private String name;
        private int age;
        private String desc;
        private int id;

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }

        public String getDesc() {
            return desc;
        }

        public void setDesc(String desc) {
            this.desc = desc;
        }

        public int getId() {
            return id;
        }

        public void setId(int id) {
            this.id = id;
        }

        @Override
        public String toString() {
            return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
        }
    }

Introduction to the AddPeopleDescProcessor.java file​

The AddPeopleDescProcessor.java file defines a class named AddPeopleDescProcessor that implements the ItemProcessor interface. This class is used to convert People objects into PeopleDESC objects.

The AddPeopleDescProcessor.java file contains the following main parts:

  1. Import other classes and interfaces.

    The file includes the following interfaces and classes:

    • People class: used to store personnel information read from the database.
    • PeopleDESC class: used to store description information after conversion or processing of personnel information.
    • ItemProcessor interface: used to process or convert the read items.

    Code:

    import com.oceanbase.example.batch.model.People;
    import com.oceanbase.example.batch.model.PeopleDESC;
    import org.springframework.batch.item.ItemProcessor;

  2. Define the AddPeopleDescProcessor class.

    The AddPeopleDescProcessor class of the ItemProcessor interface is used to convert People objects into PeopleDESC objects, implementing the logic for processing input data during batch processing.

    In the process method of this class, a PeopleDESC object desc is first created. Then, the item parameter is used to obtain the properties (name and age) of the People object, which are set to the desc object. Additionally, the desc property of the desc object is assigned a value based on the properties of the People object, generating a description. Finally, the processed PeopleDESC object is returned.

    Code:

    public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
        @Override
        public PeopleDESC process(People item) throws Exception {
            PeopleDESC desc = new PeopleDESC();
            desc.setName(item.getName());
            desc.setAge(item.getAge());
            desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
            return desc;
        }
    }

AddDescPeopleWriter.java file​

The AddDescPeopleWriter.java file implements the AddDescPeopleWriter class of the ItemWriter interface, which is used to write People objects to a database.

The AddDescPeopleWriter.java file contains the following code:

  1. Import other classes and interfaces.

    The following interfaces and classes are declared in this file:

    • PeopleDESC: a class used to store description information of a person after conversion or processing.
    • ItemWriter: an interface used to write processed or converted items to a specified target location.
    • @Autowired: an annotation used for dependency injection.
    • JdbcTemplate: a class that provides methods for executing SQL statements.
    • List: an interface used to operate on a result set.

    Sample code:

    import com.oceanbase.example.batch.model.PeopleDESC;
    import org.springframework.batch.item.ItemWriter;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.jdbc.core.JdbcTemplate;

    import java.util.List;

  2. Define the AddDescPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance. This instance is used to execute database operations when data is written.

      Sample code:

          @Autowired
          private JdbcTemplate jdbcTemplate;

    2. In the write method, traverse the input List<? extends PeopleDESC> and extract each PeopleDESC object. First, execute the SQL statement DROP TABLE people_desc to delete a table named people_desc if it exists. Then, execute the SQL statement CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255)) to create a table named people_desc with four columns: id, name, age, and description. Finally, use the SQL statement INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?) to insert the property values of each PeopleDESC object into the people_desc table.

      Sample code:

          @Override
          public void write(List<? extends PeopleDESC> items) throws Exception {
              // Drop the table if it exists
              jdbcTemplate.execute("DROP TABLE people_desc");
              // Create the table
              String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
              jdbcTemplate.execute(createTableSql);
              for (PeopleDESC item : items) {
                  String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
                  jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
              }
          }

Introduction to the AddPeopleWriter.java file​

The AddPeopleWriter.java file implements the AddDescPeopleWriter class of the ItemWriter interface, which is used to write PeopleDESC objects to a database.

The code in the AddPeopleWriter.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    Declare the following interfaces and classes in the current file:

    • People class: used to store personnel information read from the database.
    • ItemWriter interface: used to write processed or converted items to the specified target location.
    • @Autowired annotation: used for dependency injection.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • @Component annotation: used to mark the class as a Spring component.
    • List interface: used to operate on the result set.

    Sample code:

    import com.oceanbase.example.batch.model.People;
    import org.springframework.batch.item.ItemWriter;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.jdbc.core.JdbcTemplate;
    import org.springframework.stereotype.Component;

    import java.util.List;

  2. Define the AddPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance, which is used to execute database operations when writing data.

      Sample code:

          @Autowired
          private JdbcTemplate jdbcTemplate;

    2. In the write method, traverse the input List<? extends People> and extract each People object. First, execute the SQL statement DROP TABLE people to delete the people table if it exists. Then, execute the SQL statement CREATE TABLE people (name VARCHAR2(255), age INT) to create a people table with name and age columns. Finally, use the SQL statement INSERT INTO people (name, age) VALUES (?, ?) to insert the attribute values of each People object into the people table.

      Sample code:

      @Override
      public void write(List<? extends People> items) throws Exception {
          // Delete the existing table
          jdbcTemplate.execute("DROP TABLE people");
          // Create table statement
          String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
          jdbcTemplate.execute(createTableSql);
          for (People item : items) {
              String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
              jdbcTemplate.update(sql, item.getName(), item.getAge());
          }
      }

BatchConfigTest.java file​

The BatchConfigTest.java file is a class that uses JUnit to test the job configuration of Spring Batch.

The BatchConfigTest.java file contains the following code:

  1. Import other classes and interfaces.

    The following interfaces and classes are included in this file:

    • Assert class: used to assert test results.
    • Test annotation: used to mark test methods.
    • RunWith annotation: used to specify the test runner.
    • Job interface: represents a batch job.
    • JobExecution class: used to represent the execution of a batch job.
    • JobParameters class: used to represent the parameters of a batch job.
    • JobParametersBuilder class: used to build the parameters of a batch job.
    • JobLauncher interface: used to start a batch job.
    • Autowired annotation: used for dependency injection.
    • SpringBootTest annotation: used to specify the test class as a Spring Boot test.
    • SpringRunner class: used to specify the test runner as SpringRunner.

    Sample code:

    import org.junit.Assert;
    import org.junit.jupiter.api.Test;
    import org.junit.runner.RunWith;
    import org.springframework.batch.core.Job;
    import org.springframework.batch.core.JobExecution;
    import org.springframework.batch.core.JobParameters;
    import org.springframework.batch.core.JobParametersBuilder;
    import org.springframework.batch.core.launch.JobLauncher;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.test.context.junit4.SpringRunner;

    import javax.batch.runtime.BatchStatus;
    import java.util.UUID;

  2. Define the BatchConfigTest class.

    By using the SpringBootTest annotation and the SpringRunner runner, you can perform Spring Boot integration tests. In the testJob method, use the JobLauncherTestUtils helper class to start a batch job and use assertions to verify the job's execution status.

    1. Use the @Autowired annotation to automatically inject the JobLauncherTestUtils instance.

      Sample code:

      @Autowired
      private JobLauncherTestUtils jobLauncherTestUtils;

    2. Use the @Test annotation to mark the testJob method as a test method. In this method, first create a JobParameters object, then use the jobLauncherTestUtils.launchJob method to start the batch job, and use the Assert.assertEquals method to assert that the job's execution status is COMPLETED.

      Sample code:

      @Test
      public void testJob() throws Exception {
          JobParameters jobParameters = new JobParametersBuilder()
                  .addString("jobParam", "paramValue")
                  .toJobParameters();

          JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

          Assert.assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
      }

    3. Use the @Autowired annotation to automatically inject the JobLauncher instance.

      Sample code:

      @Autowired
      private JobLauncher jobLauncher;

    4. Use the @Autowired annotation to automatically inject the Job instance.

      Sample code:

      @Autowired
      private Job job;

    5. Define an internal class named JobLauncherTestUtils to help start a batch job. In this class, define a launchJob method to start the batch job. In this method, use the jobLauncher.run method to start the job and return the job's execution result.

      Sample code:

      private class JobLauncherTestUtils {
          public JobExecution launchJob(JobParameters jobParameters) throws Exception {
              return jobLauncher.run(job, jobParameters);
          }
      }

AddPeopleDescProcessorTest.java file​

The AddPeopleDescProcessorTest.java file is a class that uses JUnit to test the job configuration of Spring Batch.

The AddPeopleDescProcessorTest.java file contains the following code:

  1. Import other classes and interfaces.

    Declare the following interfaces and classes in the current file:

    • People class: stores the information of people read from the database.
    • PeopleDESC class: stores the description information of people after conversion or processing.
    • Assert class: verifies whether the expected results and actual results in the test are consistent.
    • Test annotation: marks the test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: performs dependency injection.
    • SpringBootTest annotation: specifies the test class as a Spring Boot test.
    • SpringRunner class: specifies the test runner as SpringRunner.

    Sample code:

    import com.oceanbase.example.batch.model.People;
    import com.oceanbase.example.batch.model.PeopleDESC;
    import org.junit.Assert;
    import org.junit.jupiter.api.Test;
    import org.junit.runner.RunWith;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.test.context.junit4.SpringRunner;

  2. Define the AddPeopleDescProcessorTest class.

    Use the SpringBootTest annotation and SpringRunner runner to perform integration testing of Spring Boot.

    1. Use the @Autowired annotation to automatically inject the AddPeopleDescProcessor instance.

      Sample code:

      @Autowired
      private AddPeopleDescProcessor processor;

    2. Use the @Test annotation to mark the testProcess method as a test method. In this method, first create a People object, then use the processor.process method to process the object, and assign the result to a PeopleDESC object.

      Sample code:

      @Test
      public void testProcess() throws Exception {
      People people = new People();
      people.setName("John");
      people.setAge(25);

      PeopleDESC desc = processor.process(people);
      }

File introduction of AddDescPeopleWriterTest.java​

The AddDescPeopleWriterTest.java file is a class that uses JUnit for testing the AddDescPeopleWriter write logic.

The code in the AddDescPeopleWriterTest.java file is mainly divided into the following parts:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • PeopleDESC class: Used to store descriptions that are converted or processed from information about people.
    • The Assert class: used to assert the test results.
    • @Test annotation: marks the method as a test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: used for dependency injection.
    • SpringBootTest annotation: specifies the test class as a Spring Boot test class.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • SpringRunner class: Specifies the test runner as SpringRunner.
    • ArrayList class, which is used to create an empty list.
    • List interface: used to operate the result set returned by a query.

    Sample code:

    import com.oceanbase.example.batch.model.PeopleDESC;
    import org.junit.Assert;
    import org.junit.jupiter.api.Test;
    import org.junit.runner.RunWith;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.jdbc.core.JdbcTemplate;
    import org.springframework.test.context.junit4.SpringRunner;

    import java.util.ArrayList;
    import java.util.List;

  2. Define the AddDescPeopleWriterTest class.

    You can perform an integration test for Spring Boot by using the SpringBootTest annotation and the SpringRunner runner.

    1. Use the @Autowired annotation to inject instances. Use the @Autowired annotation to automatically inject instances of AddPeopleDescProcessor and JdbcTemplate.

      Sample code:

      @Autowired
      private AddDescPeopleWriter writer;
      @Autowired
      private JdbcTemplate jdbcTemplate;

    2. Insert and query test data by using @Test. Use the @Test annotation to mark the testWrite method as a test method. First, create an empty peopleDescList list and add two PeopleDESC objects to the list. Then, use the writer.write method to write the data in the list to the database. Next, use jdbcTemplate to execute a query statement and obtain the data in the people_desc table. Use an assertion statement to verify the correctness of the data. Finally, output the query result to the console and output the information indicating that the job has been completed.

      1. Insert data into the people_desc table. First, an empty list peopleDescList of the PeopleDESC class is created. Then, two PeopleDESC objects desc1 and desc2 are created, and their properties are set. The desc1 and desc2 objects are added to the peopleDescList list. The write method of writer is then called, and the peopleDescList objects are written to the people_desc table in the database. The JdbcTemplate executes a query statement SELECT COUNT(*) FROM people_desc, retrieves the number of records from the people_desc table, and assigns the result to the variable count. Finally, the Assert.assertEquals method is used to verify whether the value of count is equal to 2.

        The sample code is as follows:

           List<PeopleDESC> peopleDescList = new ArrayList<>();
           PeopleDESC desc1 = new PeopleDESC();
           desc1.setId(1);
           desc1.setName("John");
           desc1.setAge(25);
           desc1.setDesc("This is John with age 25");
           peopleDescList.add(desc1);
           PeopleDESC desc2 = new PeopleDESC();
           desc2.setId(2);
           desc2.setName("Alice");
           desc2.setAge(30);
           desc2.setDesc("This is Alice with age 30");
           peopleDescList.add(desc2);
           writer.write(peopleDescList);

           String selectSql = "SELECT COUNT(*) FROM people_desc";
           int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
           Assert.assertEquals(2, count);

      2. Output data from the people_desc table. First, use JdbcTemplate to execute the query SELECT * FROM people_desc. Then, handle the query results using a lambda expression. In the lambda expression, use methods such as rs.getInt and rs.getString to get the field values in the query result set and set them to a new PeopleDESC object. Add the new PeopleDESC object to a result list resultDesc. Then, print a line of prompt information people_desc table data:. Use a for loop to traverse each PeopleDESC object in the resultDesc list and use System.out.println to print the content of each object. Finally, print a message indicating that the job execution is complete.

        The code is as follows:

        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
           PeopleDESC desc = new PeopleDESC();
           desc.setId(rs.getInt("id"));
           desc.setName(rs.getString("name"));
           desc.setAge(rs.getInt("age"));
           desc.setDesc(rs.getString("description"));
           return desc;
        });

        System.out.println("people_desc table data:");
        for (PeopleDESC desc : resultDesc) {
           System.out.println(desc);
        }

        // Output information after the job is completed.
        System.out.println("Batch Job execution completed.");

AddPeopleWriterTest.java file description​

The AddPeopleWriterTest.java file is a class used to test the writing logic of AddPeopleWriterTest using JUnit.

The code in the AddPeopleWriterTest.java file is mainly divided into the following sections:

  1. Import other classes and interfaces.

    Declares that the file contains the following interfaces and classes:

    • People class: stores the people information retrieved from the database.
    • Test: used to mark test methods.
    • RunWith Annotation: specifies the test runner.
    • Autowired annotation: Used for dependency injection.
    • SpringBootApplication annotation: Specifies the entry class of the Spring Boot application.
    • SpringBootTest annotation: specifies that the test class is a Spring Boot test.
    • ComponentScan annotation: specifies the package or class to scan for components.
    • JdbcTemplate class: Provides methods for executing SQL statements.
    • SpringRunner class: specifies the test runner as SpringRunner.
    • ArrayList class, which is used to create an empty list.
    • The List interface: used for operating on query result sets.

    Code:

    import com.oceanbase.example.batch.model.People;
    import org.junit.jupiter.api.Test;
    import org.junit.runner.RunWith;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.context.annotation.ComponentScan;
    import org.springframework.jdbc.core.JdbcTemplate;
    import org.springframework.test.context.junit4.SpringRunner;

    import java.util.ArrayList;
    import java.util.List;

  2. Define the AddPeopleWriterTest class.

    You can perform integration testing on Spring Boot by using the SpringBootTest annotation and SpringRunner runner, and specify the package path to be scanned by using the @ComponentScan annotation.

    1. Inject the instance using the @Autowired annotation. Use the @Autowired annotation to automatically inject the addPeopleWriter and JdbcTemplate instances.

      Here is the code:

      @Autowired
      private AddPeopleWriter addPeopleWriter;
      @Autowired
      private JdbcTemplate jdbcTemplate;

    2. Use @Test to test the insertion and output of test data.

      1. Insert data into the people table. First, create an empty People object list peopleList. Then, create two People objects person1 and person2 and set their name and age attributes. Next, add these People objects to the peopleList list. Finally, call the write method of the addPeopleWriter object, passing peopleList as a parameter, to write these People objects to the database.

        Sample code:

           List<People> peopleList = new ArrayList<>();
           People person1 = new People();
           person1.setName("zhangsan");
           person1.setAge(27);
           peopleList.add(person1);
           People person2 = new People();
           person2.setName("lisi");
           person2.setAge(35);
           peopleList.add(person2);
           addPeopleWriter.write(peopleList);

      2. Output data from the people table. The following code block uses JdbcTemplate to execute a query statement SELECT * FROM people and uses a lambda expression to process the query results. In the lambda expression, the rs.getString and rs.getInt methods are used to obtain field values from the query result set, and these values are then set in a newly created People object. Each newly created People object is added to a result list result. A prompt message people table data: is then printed, and a for loop is used to traverse each People object in the result list. Each object's content is then printed using System.out.println. Finally, an information message indicating that the task is completed is printed.

        The sample code is as follows:

           List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
               People person = new People();
               person.setName(rs.getString("name"));
               person.setAge(rs.getInt("age"));
               return person;
           });

           System.out.println("people table data:");
           for (People person : result) {
               System.out.println(person);
           }

           // Output information about when a job is completed.
           System.out.println("Batch Job execution completed.");

Full code example​

pom.xml​

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.11</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.oceanbase</groupId>
    <artifactId>java-oceanbase-springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>java-oceanbase-springbatch</name>
    <description>Demo project for Spring Batch</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.oceanbase</groupId>
            <artifactId>oceanbase-client</artifactId>
            <version>2.4.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>javax.activation-api</artifactId>
            <version>1.2.0</version>
        </dependency>
        <dependency>
            <groupId>jakarta.persistence</groupId>
            <artifactId>jakarta.persistence-api</artifactId>
            <version>2.2.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

application.properties​

#configuration database

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=

# JPA
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

# Spring Batch
spring.batch.job.enabled=false

#
logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

BatchApplication.java​

package com.oceanbase.example.batch;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class BatchApplication {
    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class, args);
    }

    public void runBatchJob() {
    }
}

BatchConfig.java​

package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;
//import javax.activation.DataSource;

@Configuration
@EnableBatchProcessing
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
@EnableAutoConfiguration
public class BatchConfig {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private DataSource dataSource;// Use the default dataSource provided by Spring Boot auto-configuration

    @Bean
    public ItemReader<People> peopleReader() {
        JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
        reader.setDataSource((javax.sql.DataSource) dataSource);
        reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
        reader.setSql("SELECT * FROM people");
        return reader;
    }

    @Bean
    public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
        return new AddPeopleDescProcessor();
    }

    @Bean
    public ItemWriter<PeopleDESC> addDescPeopleWriter() {
        return new AddDescPeopleWriter();
    }

    @Bean
    public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                      ItemWriter<PeopleDESC> writer) {
        return stepBuilderFactory.get("step1")
                .<People, PeopleDESC>chunk(10)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

    @Bean
    public Job importJob(Step step1) {
        return jobBuilderFactory.get("importJob")
                .incrementer(new RunIdIncrementer())
                .flow(step1)
                .end()
                .build();
    }
}

People.java​

package com.oceanbase.example.batch.model;

public class People {
    private String name;
    private int age;

        // getters and setters

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
    @Override
    public String toString() {
        return "People [name=" + name + ", age=" + age + "]";
    }
    // Getters and setters
}

PeopleDESC.java​

package com.oceanbase.example.batch.model;

public class PeopleDESC {
    private String name;
    private int age;
    private String desc;
    private int id;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getDesc() {
        return desc;
    }

    public void setDesc(String desc) {
        this.desc = desc;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public String toString() {
        return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
    }
}

AddPeopleDescProcessor.java​

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;


public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

AddDescPeopleWriter.java​

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

public class AddDescPeopleWriter implements ItemWriter<PeopleDESC> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table if it exists.
        jdbcTemplate.execute("DROP TABLE people_desc");
        // Create the table.
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }
}

AddPeopleWriter.java​

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class AddPeopleWriter implements ItemWriter<People> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends People> items) throws Exception {
        // Drop the table if it exists.
        jdbcTemplate.execute("DROP TABLE people");
        // Create the table.
        String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
        jdbcTemplate.execute(createTableSql);
        for (People item : items) {
            String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
            jdbcTemplate.update(sql, item.getName(), item.getAge());
        }
    }
}

BatchConfigTest.java​

package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import org.springframework.test.context.junit4.SpringRunner;

import javax.annotation.Resource;
import javax.batch.runtime.BatchStatus;
import java.util.UUID;

@RunWith(SpringRunner.class)
@SpringBootTest
public class BatchConfigTest {

    @Test
    public void testJob() throws Exception {
        JobParameters jobParameters = new JobParametersBuilder()
                .addString("jobParam", UUID.randomUUID().toString())
                .toJobParameters();

        JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();
        JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);
        Assert.assertEquals(BatchStatus.COMPLETED.toString(), jobExecution.getStatus().toString());
    }

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job job;

    private class JobLauncherTestUtils {

        public JobExecution launchJob(JobParameters jobParameters) throws Exception {
            return jobLauncher.run(job, jobParameters);
        }
    }
}

AddPeopleDescProcessorTest.java​

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddPeopleDescProcessorTest {
    @Autowired
    private AddPeopleDescProcessor processor;

    @Test
    public void testProcess() throws Exception {
        People people = new People();
  //      people.setName("John");
  //      people.setAge(25);

        PeopleDESC desc = processor.process(people);

//      Assert.assertEquals("John", desc.getName());
//        Assert.assertEquals(25, desc.getAge());
 //       Assert.assertEquals("This is John with age 25", desc.getDesc());
    }
}

AddDescPeopleWriterTest.java​

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddDescPeopleWriterTest {
    @Autowired
    private AddDescPeopleWriter writer;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {

        // Insert data into the people_desc table
        List<PeopleDESC> peopleDescList = new ArrayList<>();
        PeopleDESC desc1 = new PeopleDESC();
        desc1.setId(1);
        desc1.setName("John");
        desc1.setAge(25);
        desc1.setDesc("This is John with age 25");
        peopleDescList.add(desc1);
        PeopleDESC desc2 = new PeopleDESC();
        desc2.setId(2);
        desc2.setName("Alice");
        desc2.setAge(30);
        desc2.setDesc("This is Alice with age 30");
        peopleDescList.add(desc2);
        writer.write(peopleDescList);

        String selectSql = "SELECT COUNT(*) FROM people_desc";
        int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
        Assert.assertEquals(2, count);

        // Output the data in the people_desc table.
        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
            PeopleDESC desc = new PeopleDESC();
            desc.setId(rs.getInt("id"));
            desc.setName(rs.getString("name"));
            desc.setAge(rs.getInt("age"));
            desc.setDesc(rs.getString("description"));
            return desc;
        });

        System.out.println("people_desc table:");
        for (PeopleDESC desc : resultDesc) {
            System.out.println(desc);
        }

        // Output the information after the job execution is completed.
        System.out.println("Batch Job execution completed.");
    }
}

AddPeopleWriterTest.java​

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
public class AddPeopleWriterTest {

    @Autowired
    private AddPeopleWriter addPeopleWriter;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {
        // Insert data into the people table.
        List<People> peopleList = new ArrayList<>();
        People person1 = new People();
        person1.setName("zhangsan");
        person1.setAge(27);
        peopleList.add(person1);
        People person2 = new People();
        person2.setName("lisi");
        person2.setAge(35);
        peopleList.add(person2);
        addPeopleWriter.write(peopleList);

        // Query and output the result.
        List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
            People person = new People();
            person.setName(rs.getString("name"));
            person.setAge(rs.getInt("age"));
            return person;
        });

        System.out.println("people table data:");
        for (People person : result) {
            System.out.println(person);
        }

        // Output the information after the job is completed.
        System.out.println("Batch Job execution completed.");
    }
}

References​

For more information about OceanBase Connector/J, see OceanBase JDBC driver.

Contents

Edit this page