PARQUET-2420 : ThriftParquetWriter Convert byte to LogicalType int32 bitWidth 8 instead of primitive int32

[shreyas-dview]

The current implementation of Parquet serialisation from Thrift Definitions results in the incorrect conversion of Thrift byte fields into INT32 without preserving the required LogicalType Metadata in the Parquet file. The correct conversion should result in INT32 with LogicalType of bitWidth 8 and signed = true.

Make sure you have checked all steps below.

Jira

• My PR addresses the following Parquet Jira issues and references
  them in the PR title.
  • https://issues.apache.org/jira/browse/PARQUET-2420

Tests

• My PR adds the following unit tests

Commits

• My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines
  from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Style

• My contribution adheres to the code style guidelines and Spotless passes.
  • To apply the necessary changes, run mvn spotless:apply -Pvector-plugins

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain Javadoc that explain what it does

[shreyas-dview]

cc: @Fokko @wgtmac

[shreyas-dview]

Thrift Definition

struct TestLogicalType {
1: required i16 test_i16,
2: required byte test_i8
} 

Current Parquet Schema Generated

message ParquetSchema {
  required int32 test_i16 (INTEGER(16,true)) = 1;
  required int32 test_i8 = 2;
} 

Expected Parquet Schema

message ParquetSchema {
  required int32 test_i16 (INTEGER(16,true)) = 1;
  required int32 test_i8 (INTEGER(8,true)) = 2;
}

[Fokko]

This looks good to me, thanks!

[shreyas-dview]

@Fokko / @wgtmac Can we merge this ?

[wgtmac]

I just merged this. Thanks @shreyas-dview and @Fokko!