22390

How to read tensorflow memory mapped graph file in android?

Using Tensorflow 1.0.1 it's fine to read optimized graph and quantized graph in android using TensorFlowImageClassifier.create method, such as:

classifier = TensorFlowImageClassifier.create( c.getAssets(), MODEL_FILE, LABEL_FILE, IMAGE_SIZE, IMAGE_MEAN, IMAGE_STD, INPUT_NAME, OUTPUT_NAME);

But according to the Peter Warden's Blog(https://petewarden.com/2016/09/27/tensorflow-for-mobile-poets/), it's recommended to use memory mapped graph in mobile to avoid memory related crashes.

I built memmapped graph using

bazel-bin/tensorflow/contrib/util/convert_graphdef_memmapped_format \ --in_graph=/tf_files/rounded_graph.pb \ --out_graph=/tf_files/mmapped_graph.pb

and it created fine, but when I tried to load the file with TensorFlowImageClassifier.create(...) it says the file is not valid graph file.

In iOS, it's ok to load the file with

LoadMemoryMappedModel( model_file_name, model_file_type, &tf_session, &tf_memmapped_env);

for it has a method for read memory mapped graph.

So, I guess there's a similar function in android, but I couldn't find it.

<strong>Could someone guide me how to load memory mapped graph in android ?</strong>

Answer1:

Since the file from the memmapped tool is no longer a standard GraphDef protobuf, you need to make some changes to the loading code. You can see an example of this in the iOS Camera demo app, the LoadMemoryMappedModel() function: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/ios_examples/camera/tensorflow_utils.mm#L159

The same code (with the Objective C calls for getting the filenames substituted) can be used on other platforms too. Because we’re using memory mapping, we need to start by creating a special TensorFlow environment object that’s set up with the file we’ll be using:

std::unique_ptr<tensorflow::MemmappedEnv> memmapped_env; memmapped_env->reset( new tensorflow::MemmappedEnv(tensorflow::Env::Default())); tensorflow::Status mmap_status = (memmapped_env->get())->InitializeFromFile(file_path);

You then need to pass in this environment to subsequent calls, like this one for loading the graph.

tensorflow::GraphDef tensorflow_graph; tensorflow::Status load_graph_status = ReadBinaryProto( memmapped_env->get(), tensorflow::MemmappedFileSystem::kMemmappedPackageDefaultGraphDef, &tensorflow_graph);

You also need to create the session with a pointer to the environment you’ve created:

tensorflow::SessionOptions options; options.config.mutable_graph_options() ->mutable_optimizer_options() ->set_opt_level(::tensorflow::OptimizerOptions::L0); options.env = memmapped_env->get(); tensorflow::Session* session_pointer = nullptr; tensorflow::Status session_status = tensorflow::NewSession(options, &session_pointer);

One thing to notice here is that we’re also disabling automatic optimizations, since in some cases these will fold constant sub-trees, and so create copies of tensor values that we don’t want and use up more RAM. This setup also means it's hard to use a model stored as an APK asset in Android, since those are compressed and don't have normal filenames. Instead you'll need to copy your file out of an APK onto a normal filesytem location.

Once you’ve gone through these steps, you can use the session and graph as normal, and you should see a reduction in loading time and memory usage.

Recommend

  • Multiple statements if condition is true in shorthand if
  • Store a sequence of specific class types in Scala?
  • Viewing MJPEG Video Streams
  • Could be Text orientation of JTextArea changed by keyboard shortcut?
  • Delete file in internal memory from Android device?
  • java.lang.String cannot be converted to org.slf4j.Marker
  • Use query params of parent route to refresh the model in subroute
  • cudaFree is not freeing memory
  • Use of qualified name in function parameter
  • ggplot2 polygon world map centred with limits gives funny edges
  • Manipulating matrices in C through R
  • How to separate filename from path? basename() versus preg_split() with array_pop()
  • Defining a method whose return type is the singleton type of an argument of that method
  • DNS Lookup failed - Error with all browsers [closed]
  • How to change default stop edit behavior in jtable
  • Returning this from a constructor function in JS
  • Cursor in wrong place in contenteditable
  • Client side validation mvc dropdown
  • Cordova Apache wrong module path
  • Prolog Ambiguous Output
  • Separate ID and Class for JS and CSS
  • PushKit for VOIP iOS apps
  • Facebook iOS SDK Not Calling Completion Handler
  • netsh acl setting (need alternative method - registry settings?)
  • how to set variables in a php include file?
  • Symfony 2. CSRF token is invalid
  • Jenkins: FATAL: Could not initialize class hudson.util.ProcessTree$UnixReflection
  • Unable to decode certificate at client new X509Certificate2()
  • Custom Tabgroup Appcelerator
  • Recording logins for password protected directories
  • Apache 2.4 and php-fpm does not trigger apache http basic auth for php pages
  • JavaScriptCore crash on iOS9
  • Release, debug version and Authorization Google?
  • Why winpcap requires both .lib and .dll to run?
  • Run Powershell script from inside other Powershell script with dynamic redirection to file
  • Is there a mandatory requirement to switch app.yaml?
  • Load html files in TinyMce
  • How can I get HTML syntax highlighting in my editor for CakePHP?
  • coudnt use logback because of log4j
  • Exception on Android 4.0 `android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode)`