30293

Keras sequence prediction with multiple simultaneous sequences

Question:

My question is very similar to what it seems <a href="https://stackoverflow.com/questions/39758190/time-series-prediction-with-keras-and-multiple-sequences" rel="nofollow">this post</a> is asking, although that post doesn't pose a satisfactory solution. To elaborate, I am currently using keras with tensorflow backend and a sequential LSTM model. The end goal is I have n time-dependent sequences with equal time steps (the same number of points on each sequence and the points are all the same time apart) and I would like to feed all n sequences into the same network so it can use correlations between the sequences to better predict the next step for each sequence. My ideal output would be an n-element 1-D array with array[0] corresponding to the next-step prediction for sequence_1, array[1] for sequence_2, and so on.

My inputs are sequences of single values, so each of n inputs can be parsed into a 1-D array.

I was able to get a working model for each sequence independently using the code at the end of <a href="http://www.jakob-aungiers.com/articles/a/LSTM-Neural-Network-for-Time-Series-Prediction" rel="nofollow">this guide</a> by Jakob Aungiers, although my difficulty is adapting it to accept multiple sequences at once and correlate between them (i.e. be analyzed in parallel). I believe the issue is related to the shape of my input data, which is currently in the form of a 4-D numpy array because of how Jakob's Guide splits the inputs into sub-sequences of 30 elements each to analyze incrementally, although I could also be completely missing the target here. My code (which is mostly Jakob's, not trying to take credit for anything that isn't mine) presently looks like <a href="https://github.com/fchollet/keras/issues/8157" rel="nofollow">this</a>:

As-is this complains with "ValueError: Error when checking target: expected activation_1 to have shape (None, 4) but got array with shape (4, 490)", I'm sure there are plenty of other issues but I'd love some direction on how to achieve what I'm describing. Anything stick out immediately to anyone? Any help you could give will be greatly appreciated.

Thanks!

-Eric

Answer1:

Keras is already prepared to work with batches containing many sequences, there is no secret at all.

There are two possible approaches, though:

<ul><li>You input your entire sequences (all steps at once) and predict n results </li> <li>You input only one step of all sequences and predict the next step in a loop</li> </ul><h3>Suppose:</h3> nSequences = 30 timeSteps = 50 features = 1 #(as you said: single values per step) outputFeatures = 1 <h3>First apporach: stateful=False:</h3> inputArray = arrayWithShape((nSequences,timeSteps,features)) outputArray = arrayWithShape((nSequences,outputFeatures)) input_shape = (timeSteps,features) #use layers like this: LSTM(units) #if the first layer in a Sequential model, add the input_shape #if you want to return the same number of steps (like a new sequence parallel to the input, use return_sequences=True

Train like this:

model.fit(inputArray,outputArray,....)

Predict like this:

newStep = model.predict(inputArray) <h3>Second approach: stateful=True:</h3> inputArray = sameAsBefore outputArray = inputArray[:,1:] #one step after input array inputArray = inputArray[:,:-1] #eliminate the last step batch_input = (nSequences, 1, features) #stateful layers require the batch size #use layers like this: LSMT(units, stateful=True) #if the first layer in a Sequential model, add input_shape

Train like this:

model.reset_states() #you need this in stateful=True models #if you don't reset states, #the stateful model will think that your inputs are new steps of the same previous sequences for step in range(inputArray.shape[1]): #for each time step model.fit(inputArray[:,step:step+1], outputArray[:,step:step+1],shuffle=False,...)

Predict like this:

model.reset_states() predictions = np.empty(inputArray.shape) for step in range(inputArray.shape[1]): #for each time step predictions[:,step] = model.predict(inputArray[:,step:step+1])

Recommend

  • Deleting document by Term from lucene
  • Android udp packets loss
  • How can I define Settings and Analyzer for ElasticSearch?
  • Elasticsearch Reindex or Flag Deleted Type Property
  • init method of a Serlvet without ServletException
  • Keras sequence prediction with multiple simultaneous sequences
  • Index Analyzed and not Analyzed?
  • SQL Server Report Builder: Hiding Tablix Rows based on value in Subreport
  • Switching from VS debugger to WinDbg
  • Inorder traversal in lisp
  • Visual Studio freezes on breakpoints
  • Lucene updateDocument not deleting documents
  • Log a web page's dynamically-created tag attributes with a userscript
  • Cutting down bag to pass to udf
  • Performance implications of a large number of mutexes
  • Why postgresql does not use index in this query
  • How to limit the increment of variable in mysql
  • Detect Removable Media Ejection Request in Windows Service
  • Regarding RandomTree in Weka
  • Algorithm to determine thread “hotness”
  • looking for a slight variant of GROUP BY
  • iPython magic for Zipline cannot find data bundle
  • How can I get the maximum number of OpenMP threads that may be created during the whole execution of
  • Ray-tracing triangles
  • How to start server for Selenium grid Java Maven setup
  • Number of threads being used during Parallel.ForEach
  • Most efficient way to replace lowest list values in dataframe in R
  • Yii2: Finding file and getting path in a directory tree
  • Clear activity stack before launching another activity
  • Angular2 Response for preflight is invalid (redirect) from some GET requests
  • How do I configure context broker accept post requests from my remote sensor?
  • Repeat a vertical line on every page in Report Builder / SSRS
  • Display issues when we change from one jquery mobile page to another in firefox
  • Android screen density dpi vs ppi
  • Different response to non-authenticated users and AJAX calls
  • Bug in WPF DataGrid
  • Finding past revisions of files in StarTeam w/ .NET SDK / C#
  • Arrow is showed instead of the material design version hamburger icon. Why doesn't syncState in
  • Data Validation Drop Down Box Arrow Disappearing
  • java string with new operator and a literal