Skip to content

v0.5.2
Compare
Choose a tag to compare
@JoshRosen JoshRosen released this 23 Oct 17:37
· 81 commits to master since this release
v0.5.2
3e32744

spark-redshift 0.5.2 is a maintenance release that contains a handful of important bugfixes. We recommend that all users upgrade to this release.

Bug Fixes:

  • Fixed a thread-safety issue which could lead to errors or data corruption when processing date, timestamp, or decimal columns (#107 / #108).
  • Fixed bugs related to handling of S3 credentials when they are specified as part of the tempdir URL (#109).
  • Fixed a typo in the AWS credentials section of the README: the old text referred to sc.hadoopConfig instead of sc.hadoopConfiguration (#109).

Enhancements:

  • Added a new extracopyoptions configuration, which allows advanced users to pass additional options to Redshift in COPY commands (#35).
  • Added an example of writing data back to Redshift using the SQL language API (#110).
  • Added documentation on how to configure the SparkContext's global hadoopConfiguration from Python (#109).
  • Added a tutorial (#101 and #106).
Assets 2
Loading