30192

How to change string columns size for all columns in pandas hdfstore?

Question:

How can the min_itemsize for <strong>all</strong> string columns be changed in a hdf5 table? I don't know my dataframe structure during run time, and thus can not hardcode it.

Answer1:

see the docs <a href="http://pandas.pydata.org/pandas-docs/dev/io.html#string-columns" rel="nofollow">here</a>.

The itemsize is created on the first append (and cannot be changed later). If min_itemsize is not specified it will be the max length of strings in that append.

In [1]: df = DataFrame({ 'A' : ['foo','bar']}) In [2]: store = pd.HDFStore('test.h5',mode='w') In [3]: store.append('df',df,min_itemsize=30) In [4]: store.get_storer('df') Out[4]: frame_table (typ->appendable,nrows->2,ncols->1,indexers->[index]) In [5]: store.get_storer('df').table Out[5]: /df/table (Table(2,)) '' description := { "index": Int64Col(shape=(), dflt=0, pos=0), "values_block_0": StringCol(itemsize=30, shape=(1,), dflt='', pos=1)} byteorder := 'little' chunkshape := (1724,) autoindex := True colindexes := { "index": Index(6, medium, shuffle, zlib(1)).is_csi=False} In [8]: store['df'] Out[8]: A 0 foo 1 bar In [6]: store.close()

Recommend

  • Transpose table then set and rename index
  • iOS 6 dateFromString returns wrong date
  • Checking a play current mode makes an error occur
  • Fat binaries in iOS
  • Building jamvm 1.5.4 on OS X Lion
  • runtime error when linking ffmpeg libraries in qt creator
  • several dataProvider per one Test in TestNG
  • How to retrieve information from antrun back to maven?
  • Getting media player state in windows phone 7
  • R convert summary result (statistics with all dataframe columns) into dataframe
  • How to specify input and output paths from cmd.exe for a PowerShell script?
  • Make new pandas columns based on pipe-delimited column with possible repeats
  • RxJava debounce by arbitrary value
  • D3 get axis values on zoom event
  • Breaking out column by groups in Pandas
  • What does 'Language neutral' mean with regard to MAKELANGID?
  • xtable package: Skipping some rows in the output
  • C: Incompatible pointer type initializing
  • How do I get HTML corresponding to current DOM tree?
  • Meteor helpers not available in Angular template
  • R - Combining Columns to String Based on Logical Match
  • How do I change content of ComboFieldEditor?
  • Align navbar back button on right side
  • Window Size for Mac application
  • How to show dropdown in excel using jrxml (jasper api)?
  • When should I choose bucket sort over other sorting algorithms?
  • Build own AppleScript numerical error handling
  • Weird JavaScript statement, what does it mean?
  • Do I've to free mysql result after storing it?
  • How to get next/previous record number?
  • VB.net deserialize, JSON Conversion from type 'Dictionary(Of String,Object)' to type '
  • retrieve vertices with no linked edge in arangodb
  • SQL merge duplicate rows and join values that are different
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • How to CLICK on IE download dialog box i.e.(Open, Save, Save As…)
  • Change div Background jquery
  • apache spark aggregate function using min value
  • reshape alternating columns in less time and using less memory
  • java string with new operator and a literal
  • How to push additional view controllers onto NavigationController but keep the TabBar?