FileVerificationSkipper for Spring batch FlatFile Reader

import java.io.FileNotFoundException; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; import java.nio.file.StandardOpenOption; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.batch.core.step.skip.SkipLimitExceededException; import org.springframework.batch.core.step.skip.SkipPolicy; import org.springframework.batch.item.file.FlatFileParseException; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.PropertySource; /** * The Class FileVerificationSkipper. * * @author RAVI VARMA YARAkARAJU */ @PropertySource("classpath:config.properties") public class FileVerificationSkipper implements SkipPolicy { private static final Logger logger = LoggerFactory.getLogger(FileVerificationSkipper.class); @Value("${datafile.errorFile}") private String errorFile; /** * Returns true or false, indicating whether or not processing should * continue with the given throwable. Clients may use * {@code skipCount<0} to probe for exception types that are skippable, * so implementations should be able to handle gracefully the case where * {@code skipCount<0}. Implementations should avoid throwing any * undeclared exceptions. * * @param exception exception encountered while reading * @param skipCount currently running count of skips * @return true if processing should continue, false otherwise. * @throws SkipLimitExceededException if a limit is breached * @throws IllegalArgumentException if the exception is null * * This method also write error records into error log file. * */ @Override public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException { if (exception instanceof FileNotFoundException) { return false; } else if (exception instanceof FlatFileParseException && skipCount <= 15) { FlatFileParseException ffpe = (FlatFileParseException) exception; StringBuilder errorMessage = new StringBuilder(); errorMessage.append("An error occured while processing the " + ffpe.getLineNumber() + " line of the file. Below was the faulty " + "input.\n"); errorMessage.append(ffpe.getInput() + "\n"); //logger.error("{}", errorMessage.toString()); try { Path path = Paths.get(errorFile); if (Files.notExists(path)){ Files.createFile(path); } Files.write(Paths.get(errorFile), errorMessage.toString().getBytes(), StandardOpenOption.APPEND); } catch (IOException e) { e.printStackTrace(); } return true; } else { return false; } } }
With the skip technique you may specify certain exception types and a maximum number of skipped items, and whenever one of those skippable exceptions is thrown, the batch job doesn’t fail but skip the item and goes on with the next one. Only when the maximum number of skipped items is reached, the batch job will fail. For example, Spring Batch provides the ability to skip a record when a specified Exception is throw when there is an error reading a record from your input. This section will look at how to use this technique to skip records based upon specific Exceptions.

This also stores error record in separate file for later verification.

Be the first to comment

You can use [html][/html], [css][/css], [php][/php] and more to embed the code. Urls are automatically hyperlinked. Line breaks and paragraphs are automatically generated.