I read online that in PyArrow a string column would have a column-level size limit of 2GB. However, in my work I noticed this doesn't hold. def some_function( self, raw_table: pa.Table, ): schema = ...
Python CSV Column Remover Script A Python script that allows you to remove specified columns from CSV files.