1893

Issue deserializing (protocolBuffer) serialized data using protobuf-net

Question:

I serialized data using protobuf-net, and am able to deser the same in C#.

putting a C# dummy w#

var file = File.Create("animal.bin"); //Creating Msg - Fill the Data animal.id = "1"; animal.Name = "Rat"; animal.host = "Cheetha"; ProtoBuf.Serializer.SerializeWithLengthPrefix(file, animal, PrefixStyle.Base128, 1); animal.id = "2"; animal.Name = "Cat"; animal.host = "Cheetha"; ProtoBuf.Serializer.SerializeWithLengthPrefix(file, animal, PrefixStyle.Base128, 1); .... animal.id = "4"; animal.name = "Cheetha"; animal.host = "Cheetha"; ProtoBuf.Serializer.SerializeWithLengthPrefix(file, animal, PrefixStyle.Base128, 1); //Done Creating Msg file.Close();

So far so good... No Issues here. But when I try to deserialize the same in C++ using protocol-buffer, I am unable to get the correct data

cpp code...

GOOGLE_PROTOBUF_VERIFY_VERSION; string fpath = "animal.bin"; fstream input(fpath, ios::in | ios::binary); if (!input) { cerr << "failed to open " << fpath << endl; return false; } ZeroCopyInputStream *raw_in = new IstreamInputStream(&input); CodedInputStream *coded_in = new CodedInputStream(raw_in); google::protobuf::uint32 n; std::string tmpStr; animal::animalInfo animalList; coded_in->ReadVarint32(&n); cout << "# " << n << endl; //output: #10 coded_in->ReadRaw(&tmpStr,n); //tmpStr shows data like >>1..Rat..Ch animalList.ParseFromArray(&tmpStr,1);//Not sure if this is correct usage?

I am sure I am making a mistake but not able to understand whats wrong.... Have read and reread lots of post on this but dont see whats still wrong

Using Protocol Buffer2.5, protobuf-netR622, Visual Studio 2010

Answer1:

I think you're just mismatching the headers; a length-prefix <em>and</em> a field-header (of field 1) is <strong>two</strong> "varint"s; the <em>first</em> "varint" will always be decimal 10 (10 means: field 1, length-prefixed). The <em>second</em> "varint" tells you the length of the next data. So - if you want to decode it <em>manually</em> you would call ReadVarint32 a <em>second</em> time. I'm not familiar with ReadRaw, but if the second parameter is the number of bytes to read, then <em>it goes there</em>, i.e.

coded_in->ReadVarint32(&n); // field header // assert: n === 10 coded_in->ReadVarint32(&n); // length coded_in->ReadRaw(&tmpStr,n);

Alternatively, just use a wrapper object - i.e.

message animals { repeated animal items = 1; }

and deserialize it <em>as an instance of animals</em> - this uses the exact same layout. The only difference here is that it will load <em>all</em> the items in one go - so it might be problematic if you are reading very long streams.

Another alternative would be: <em>don't add a field-header</em>:

Serializer.SerializeWithLengthPrefix(file, animal, PrefixStyle.Base128, 0);

Then you would only read one "varint":

coded_in->ReadVarint32(&n); // length coded_in->ReadRaw(&tmpStr,n);

Recommend

  • renaming file extensions in android
  • Python paramiko logging messes up stfp connection
  • Boost.Filesystem create_directories mangles directory name
  • Python paramiko logging messes up stfp connection
  • C++ Finding Anagrams in words
  • ReadFile only reading one buffer character?
  • C++ Boost graph library: Building a vector of vertices visited in an undirected graph search?
  • C++ Copy directory recursive under unix
  • Junk characters while reading text file in java
  • got OSError when use pillow to deal with my jpg image
  • C++ Boost ptr_map serialization error
  • Backward compatibility of Python 3.5 for external modules
  • Facebook iOS SDK Not Calling Completion Handler
  • Hibernate to update table schema
  • CERN ROOT exporting data to plain text
  • URLConnection doesn't work since API 10 and higher?
  • Web.config system.webserver errors
  • Jenkins: FATAL: Could not initialize class hudson.util.ProcessTree$UnixReflection
  • Problems installing Yesod for Haskell
  • Unable to install Git-core+svn by MacPorts
  • Unable to decode certificate at client new X509Certificate2()
  • Abort upload large uploads after reading headers
  • Custom Tabgroup Appcelerator
  • VSO Build — Response status code does not indicate success: 404 (Not Found)
  • The plugin 'org.apache.maven.plugins:maven-jboss-as-plugin' does not exist or no valid ver
  • ilmerge with a PFX file
  • Java Scanner input dilemma. Automatically inputs without allowing user to type
  • Launch Runnable Jar from Web Start
  • JavaScriptCore crash on iOS9
  • what is the difference between the asp.net mvc application and asp.net web application
  • How do you troubleshoot character encoding problems?
  • Function pointer “assignment from incompatible pointer type” only when using vararg ellipsis
  • 0x202A in filename: Why?
  • Matrix multiplication with MKL
  • File not found error Google Drive API
  • costura.fody for a dll that references another dll
  • Binding checkboxes to object values in AngularJs
  • Observable and ngFor in Angular 2
  • UserPrincipal.Current returns apppool on IIS
  • java string with new operator and a literal