11235

Caffe, setting custom weights in layer

I have a network. In one place I want to use concat. As on this picture. <img src=https://www.e-learn.cn/content/wangluowenzhang/"https://i.stack.imgur.com/wXON3.png" alt="picture">

Unfortunately, the network doesn't train. To understand why I want to change weights in concat. Meaning that all values from FC4096 will get 1 and all values from FC16000 will get 0 at the beginning.

I know that FC4096 will get me 57% accuracy, so with learning rate 10^-6 I will understand why after concatenation layers didn't learn.

The question is, how can I set all values from FC4096 to 1 and all values from FC16000 to 0?

Answer1:

You can add a "Scale" layer on top of FC16000 and init it to 0:

layer { name: "scale16000" type: "Scale" bottom: "fc16000" top: "fc16000" # not 100% sure this layer can work in-place, worth trying though. scale_param { bias_term: false filler: { type: "constant" value: 0 } } param { lr_mult: 0 decay_mult: 0 } # set mult to non zero if you want to train this scale }

Recommend

  • Specify some untrainable filters for Keras convolutional network
  • call constructor of class with brace enclosed initilizer list
  • how to create a padding around the edge of a google map
  • MiniProfiler with multilayered WCF services
  • How to tackle diagonally stacked, rounded image background element hovers?
  • Create domain with matrices in Chapel
  • You must feed a value for placeholder tensor 'Placeholder' with dtype float(Tensorflow)
  • concatenating results from SQL query and NULL columns
  • Display an Image using C# in Web Application
  • Clustering algorithm with different epsilons on different axes
  • Why is the 100*0.07 equal to 6.9999…? [duplicate]
  • 'SOCK_RAW' option in 'socket' system call
  • Regex is not working as expected?
  • Ordering of eigenvectors when calculating eigenvectors using LAPACK's ssteqr
  • python function to set accuracy of float
  • Howto count a variable inside of innerHTML?
  • What's a better way to swap two argument values?
  • How to use a decaying learning rate with an estimator in tensorflow?
  • use rvest and css selector to extract table from scraped search results
  • SQL Server: +(unary) operator on non-numeric Strings
  • ValueError: Found arrays with inconsistent numbers of samples
  • How to get file download speed (transfer rate) with php?
  • Connect Node.js with Oracle on Windows platform
  • .NET video play library which allows to change the playback rate?
  • Row to Column conversion in Talend
  • Custom preprocessing in caret
  • calculate gradient output for Theta update rule
  • Is it possible to open regedit and navigate to straight to a specific key using process.start?
  • Mysql - How to search for 26 records that each begins with the letter of the alphabet?
  • Alternative to overridePendingTransition() - Android
  • preg_replace Double Spaces to tab (\\t) at the beginning of a line
  • formatting the colorbar ticklabels with SymLogNorm normalization in matplotlib
  • Xamarin Forms - UWP Fonts
  • Update CALayer sublayers immediately
  • Arrow is showed instead of the material design version hamburger icon. Why doesn't syncState in
  • Arrays break string types in Julia
  • Change div Background jquery
  • Qt: Run a script BEFORE make
  • reshape alternating columns in less time and using less memory
  • Net Present Value in Excel for Grouped Recurring CF