It is a very common mistake to forget that the activations, gradients and optimizer moment tracking variables also take VRRAM, not just the parameters, increasing memory usage quite a bit. The backprob calculations themselves make it so the training phase takes almost double the VRAM of forward / inference use of the neural net, and … Read more
Keras and TF themselves don’t use whole cores and capacity of CPU! If you are interested in using all 100% of your CPU then the multiprocessing.Pool basically creates a pool of jobs that need doing. The processes will pick up these jobs and run them. When a job is finished, the process will pick up … Read more
If y is a theano variable, y.shape will be a theano variable. so it is normal that print y.shape return: Shape.0 If you want to evaluate the expression y.shape, you can do: y.shape.eval() if y.shape do not input to compute itself(it depend only on shared variable and constant). Otherwise, if y depend on the x … Read more
Well, max-pooling and monotonely increasing non-linearities commute. This means that MaxPool(Relu(x)) = Relu(MaxPool(x)) for any input. So the result is the same in that case. So it is technically better to first subsample through max-pooling and then apply the non-linearity (if it is costly, such as the sigmoid). In practice it is often done the … Read more
If you want the output printed in a fancy way: model.summary() If you want the sizes in an accessible form for layer in model.layers: print(layer.get_output_at(0).get_shape().as_list()) There are probably better ways to access the shapes than this. Thanks to Daniel for the inspiration.
In keras, non-trainable parameters (as shown in model.summary()) means the number of weights that are not updated during training with backpropagation. There are mainly two types of non-trainable weights: The ones that you have chosen to keep constant when training. This means that keras won’t update these weights during training at all. The ones that … Read more
To get the gradients of model output with respect to weights using Keras you have to use the Keras backend module. I created this simple example to illustrate exactly what to do: from keras.models import Sequential from keras.layers import Dense, Activation from keras import backend as k model = Sequential() model.add(Dense(12, input_dim=8, init=”uniform”, activation=’relu’)) model.add(Dense(8, … Read more
termplotlib (a small project of mine) might come in handy here. Install with pip install termplotlib and produce terminal plots like import termplotlib as tpl import numpy as np x = np.linspace(0, 2*np.pi, 100) y = np.sin(x) + x fig = tpl.figure() fig.plot(x, y, width=60, height=20) fig.show() 7 +—————————————————+ | | 6 | ** | … Read more
This question was raised at the Keras github repository in Issue #4446: Quick Question: can a model be fit for multiple times? It was closed by François Chollet with the following statement: Yes, successive calls to fit will incrementally train the model. So, yes, you can call fit multiple times.
Create a .keras (note the . in front) folder in you home directory and put the keras.json file there. For example, /home/DaniPaniz/.keras/keras.json (or ~/.keras/keras.json in short) if you are on a UNIX like system (MacOS X, Linux, *BSD). On Windows you want to create the folder %USERPROFILE%/.keras and put the JSON file there. Alternatively, you … Read more