Question:

My question is very similar to what it seems <a href="https://stackoverflow.com/questions/39758190/time-series-prediction-with-keras-and-multiple-sequences" rel="nofollow">this post</a> is asking, although that post doesn't pose a satisfactory solution. To elaborate, I am currently using keras with tensorflow backend and a sequential LSTM model. The end goal is I have n time-dependent sequences with equal time steps (the same number of points on each sequence and the points are all the same time apart) and I would like to feed all n sequences into the same network so it can use correlations between the sequences to better predict the next step for each sequence. My ideal output would be an n-element 1-D array with array[0] corresponding to the next-step prediction for sequence_1, array[1] for sequence_2, and so on.

My inputs are sequences of single values, so each of n inputs can be parsed into a 1-D array.

I was able to get a working model for each sequence independently using the code at the end of <a href="http://www.jakob-aungiers.com/articles/a/LSTM-Neural-Network-for-Time-Series-Prediction" rel="nofollow">this guide</a> by Jakob Aungiers, although my difficulty is adapting it to accept multiple sequences at once and correlate between them (i.e. be analyzed in parallel). I believe the issue is related to the shape of my input data, which is currently in the form of a 4-D numpy array because of how Jakob's Guide splits the inputs into sub-sequences of 30 elements each to analyze incrementally, although I could also be completely missing the target here. My code (which is mostly Jakob's, not trying to take credit for anything that isn't mine) presently looks like <a href="https://github.com/fchollet/keras/issues/8157" rel="nofollow">this</a>:

As-is this complains with "ValueError: Error when checking target: expected activation_1 to have shape (None, 4) but got array with shape (4, 490)", I'm sure there are plenty of other issues but I'd love some direction on how to achieve what I'm describing. Anything stick out immediately to anyone? Any help you could give will be greatly appreciated.

Thanks!

-Eric

Answer1:Keras is already prepared to work with batches containing many sequences, there is no secret at all.

There are two possible approaches, though:

<ul><li>You input your entire sequences (all steps at once) and predict n results </li> <li>You input only one step of all sequences and predict the next step in a loop</li> </ul><h3>Suppose:</h3>```
nSequences = 30
timeSteps = 50
features = 1 #(as you said: single values per step)
outputFeatures = 1
```

<h3>First apporach: `stateful=False`

:</h3>
```
inputArray = arrayWithShape((nSequences,timeSteps,features))
outputArray = arrayWithShape((nSequences,outputFeatures))
input_shape = (timeSteps,features)
#use layers like this:
LSTM(units) #if the first layer in a Sequential model, add the input_shape
#if you want to return the same number of steps (like a new sequence parallel to the input, use return_sequences=True
```

Train like this:

```
model.fit(inputArray,outputArray,....)
```

Predict like this:

```
newStep = model.predict(inputArray)
```

<h3>Second approach: `stateful=True`

:</h3>
```
inputArray = sameAsBefore
outputArray = inputArray[:,1:] #one step after input array
inputArray = inputArray[:,:-1] #eliminate the last step
batch_input = (nSequences, 1, features) #stateful layers require the batch size
#use layers like this:
LSMT(units, stateful=True) #if the first layer in a Sequential model, add input_shape
```

Train like this:

```
model.reset_states() #you need this in stateful=True models
#if you don't reset states,
#the stateful model will think that your inputs are new steps of the same previous sequences
for step in range(inputArray.shape[1]): #for each time step
model.fit(inputArray[:,step:step+1], outputArray[:,step:step+1],shuffle=False,...)
```

Predict like this:

```
model.reset_states()
predictions = np.empty(inputArray.shape)
for step in range(inputArray.shape[1]): #for each time step
predictions[:,step] = model.predict(inputArray[:,step:step+1])
```