984

How to read a complex JSON in spring batch?

Question:

I have a complex JSON below. I am reading it using FlatFileItemReader. How can I ignore the last line "]", with my customized ComplexJsonRecordSeparatorPolicy?

[ {"firstName":"Tom", "lastName":"Cruise"}, {"firstName":"Bruce", "lastName":"Willis"}, {"firstName":"Liam", "lastName":"Neeson"} ]

My ComplexJsonRecordSeparatorPolicy looks like below. This class is successfully working, when I have "]" in line no 4 but, it throws an error when the line is supplied with only "]" in line no 5, as my post processor deletes the line instead of ignoring it.

public class ComplexJsonRecordSeparatorPolicy extends JsonRecordSeparatorPolicy { @Override public boolean isEndOfRecord(String line) { return StringUtils.countOccurrencesOf(line, "{") == StringUtils.countOccurrencesOf(line, "}") && (line.trim().endsWith("}") || line.trim().endsWith(",") || line.trim().endsWith("]")); } @Override public String postProcess(String record) { if (record.startsWith("[")) record = record.substring(1); if ((record.endsWith("]") || record.endsWith(","))) record = record.substring(0, record.length() - 1); return super.postProcess(record); } }

Answer1:

I have created a small example for this one. Please take a look and let me know your thoughts

<a href="https://github.com/bigzidane/spring-batch-jsonListItem-reader" rel="nofollow">https://github.com/bigzidane/spring-batch-jsonListItem-reader</a>.

For more information, I have created a Reader which is to parse JSON as a List and then map every entry to a POJO through 'classToBound' and then return each by each follow Spring Batch standard.

<strong>The example takes an Json file as</strong>

[ { "name": "zidane", "nation": "france" }, { "name": "ronaldo", "nation": "brazil" }, { "name": "marcelo", "nation": "brazil" } ]

<strong>The job configuration</strong>

<job id="exampleJsonReaderJob" xmlns="http://www.springframework.org/schema/batch"> <step id="stepId"> <tasklet> <chunk reader="exampleJsonReader" writer="exampleWriter" processor="exampleProcessor" commit-interval="5" /> </tasklet> </step> </job>

<strong>The Reader</strong>

<bean id="exampleJsonReader" class="com.itservicesdepot.example.springbatch.jsonreader.reader.JsonFileListItemReader" scope="step"> <property name="resource" value="classpath:soccers.json" /> <property name="classToBound" value="com.itservicesdepot.example.springbatch.jsonreader.model.SoccerJsonEntry" /> </bean>

The Processor and Writer are just normal ones.

In the reader, there is an property "classToBound", that is the Pojo which is top map with an item in Json list.

Recommend

  • Is it possible to run the MSBUILD.SONARQUBE.RUNNER on a CSProj file as opposed to a solution?
  • Input/output in GLPK for Java
  • HttpCore for measuring http request/response elapsed time
  • Unit Testing Spring MVC Controller that returns PagedResources
  • Extending a Function1 in Scala
  • Does Perl currently (5.8 and 5.10) make any promises about the order alternations will be used?
  • Permission screen appears every time
  • pyspark substring and aggregation
  • Why is the following two duplicate finder algorithms have different time complexity?
  • Conversion from C++ to Delphi (simple)
  • C# “cannot assign field because it is a foreach iteration variable”
  • Excel VBA Intersect
  • Python equivalent of Scala's exists() function?
  • Spring Boot OAuth2 SSO with access/refresh tokens is not stored in a database correctly
  • reduce/reduce conflicts using ocamlyacc
  • Better file search algorithm than creating a list of files
  • how to set id for each item in ListView
  • Excel 2007: Format of email address from Outlook 2007
  • Updating and removing unique join relationships in CakePHP
  • Azure table query partial partitionkey guid match
  • Initialization section of the package
  • calling a fragment from fragment
  • Negating Regex PO BOX
  • to implement a spinner in angular2+
  • MVC - @Html.CheckBoxFor
  • How to populate html table with info from list in django
  • Replace last two characters in column
  • List comprehension with if conditional to get list of files of a specific type
  • System.InvalidCastException: Specified cast is not valid
  • How can I extract results of aggregate queries in slick?
  • OOP Javascript - Is “get property” method necessary?
  • Marklogic : Query response time is very high
  • Why querying a date BC is changed to AD in Java?
  • Record samples being played with OpenAL
  • PHP - How to update data to MySQL when click a radio button
  • Rearranging Cells in UITableView Bug & Saving Changes
  • Circular dependency while pushing http interceptor
  • Linker errors when using intrinsic function via function pointer
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • Bitwise OR returns boolean when one of operands is nil