153 questions
1
vote
2
answers
115
views
What is the equivalent of torch.nn.Parameter(...) in julia's flux?
In pytorch I can create a custom module as follows (this code example is taken from here):
from torch import nn
class MyModel(nn.Module):
def __init__(self):
super().__init__()
...
2
votes
1
answer
94
views
MethodError in Training Minimal U-Net
I need to implement a U-Net in Julia using Flux.jl. The ultimate goal is to train a neural network for a scientific problem. As a first step, I decided to experiment with the KITTI benchmark dataset.
...
0
votes
2
answers
109
views
How to load pre-trained models without having to load Flux.jl?
I have a pre-trained neural network which I want to use it without having to load Flux.jl because it takes some seconds to load flux.jl. Should I write the model again without having to load Flux.jl?
...
0
votes
1
answer
122
views
How can I update the actor network parameters in PPO using the gradient from Flux.jl? Gradients return nothing
To preface, I am a complete Julia newbie... I am trying to implement PPO for the first time and I've been having issues updating the actor (and by extension critic) network parameters using the ...
0
votes
1
answer
88
views
Flux UndefVarError: `setup` not defined
I have the following issue when running the following code,
hparams = HyperParams()
gen = generator()
enc = encoder()
dec = decoder()
gen_opt = Flux.setup(Flux.Adam(hparams.lr_gen), gen)
enc_opt = ...
0
votes
1
answer
140
views
Accelerate/Parallelize the training ML algorithm in Flux
I am experimenting with a machine learning method. My goal is to define a loss function that can evaluate the convergence of the model towards a uniform data distribution.
I generate K random ...
1
vote
0
answers
135
views
ERROR: UndefVarError: `setup` not defined when loading setup function in flux
I am using M1 mac (if that helps) and when I tried loading the setup function flux.jl and got the following error. UndefVarError setup not defined
The flux version I am using is 0.13.4. I created a ...
1
vote
1
answer
164
views
What is a model in Julia Flux 0.13 and higher?
(also posted in Julia discourse https://discourse.julialang.org/t/what-is-a-model-in-julia-flux-0-13-and-higher/100653)
I want to use Julia Flux for machine learning with custom models (not neural ...
1
vote
1
answer
104
views
Reparametrization trick in Flux.jl
Does Flux.jl have an equivalent to rsample in PyTorch that automatically implements these stochastic/policy gradients. That way the reparameterized sample becomes differentiable.
2
votes
0
answers
66
views
Is there a way to impose a constraint on the output of a Flux neural network model?
Suppose that I have the observed data $y_i\sim N(0,1/w(x_i)), $ where $x_i=i$ for $i=1,...,100$. I define the true function $w(x_i)=(x_i-50)^2$ and hope to recover this function with a Flux neural ...
1
vote
1
answer
120
views
Stepwise decrease GPU utility followed by out of memory error
I am running a 3D image segmentation deep learning training pipeline on a GCloud VM and am noticing a stepwise decrease of the GPU utility after about 25 epochs and an out of memory error after 32 ...
0
votes
0
answers
113
views
Compile time in Julia when training a model in Flux is too high
I'm training a simple model on Julia using Flux and I'm trying to compare the training times between training on GPU and on CPU (also between Julia and Python using the same model in Python with ...
0
votes
2
answers
341
views
Same model performs very diferent in Keras and Flux
In a class I'm taking, the professor gave us two datasets, one of 301 late-type galaxies and the other one of 301 early-type galaxies, and we build a model in Keras so it can differentiate them:
...
0
votes
0
answers
97
views
Accuracy and gradient update not within the same training loop
I am following the example mlp_mnist.jl from Flux model zoo and I am a little confused with the accuracy computation and train functions.
The accuracy computation function loops over the dataloader ...
4
votes
1
answer
740
views
Julia Flux withgradient operation
I am a newbie to Julia and Flux with some experience in Tensorflow Keras and python. I tried to use the Flux.withgradient command to write a user-defined training function with more flexibility. Here ...
1
vote
1
answer
608
views
Why am I getting a "Mutating arrays is not supported" error here?
Here's a minimal working example.
using Flux
myvector = []
for i in 1:3
push!(myvector, 0.1 * i) #Produces a vector [0.1, 0.2, 0.3]
end
Q = Chain(Dense(3, 64, relu), Dense(64, 32, relu), Dense(...
3
votes
0
answers
256
views
simple example returns MethodError: no method matching Dense(::Pair{Int64, Int64})
I am running through basic examples for Flux, and I am getting MethodErrors when running code verbatim through the REPL. I am using a fresh REPL and am on VERSION 1.8.2 for linux.
julia> using ...
1
vote
0
answers
390
views
Using the GPU with Lux and NeuralPDE Julia
I am trying to run a model using the GPU, no problem with the CPU. I think somehow using measured boundary conditions is causing the issue but I am not sure. I am following this example: https://...
2
votes
1
answer
199
views
What is this syntax with a function from an empty tuple in Julia Flux gradient?
I can't find a reference documentation for the gradient function of Julia Flux, there are only several tutorial examples.
I understand how gradient is used to compute gradients of functions, e.g. the ...
3
votes
3
answers
704
views
Is there a way to print loss from Flux.train?
I'm trying to train a UNet in Julia with the help of Flux.
Flux.train!(loss, Flux.params(model), train_data_loader, opt)
batch_loss = loss(train_data, train_targets)
where the loss is
...
1
vote
0
answers
252
views
Error while using GPU on UNET with FLUX in Julia
I'm using Julia for UNet training with GPU. (Runs without issues on CPU).
The error is at NNlib's function conv_im2col()
function conv_im2col!(
y::AbstractArray{T,5}, x::AbstractArray{...
9
votes
1
answer
2k
views
What is the purpose of [extras] and [targets] in a Julia Project.toml?
In the following example (code copied from the Flux project.toml) what is the purpose of the [extras] and [targets] sections?
[compat]
Adapt = "3.0"
ArrayInterface = "3.1, 4, 5, 6"
...
0
votes
1
answer
296
views
Installing Flux.jl on Linux: NNlibCUDA fails to precompile
While trying to install Flux I run into an error where it fails to precompile the NNlibCUDA library.
Running ] test NNlibCUDA results in LoadError: UndefVarError: upsample_linear_wcn! not defined:
(@...
2
votes
1
answer
421
views
Error in the most simplest example in Flux.jl
I am testing the example here: https://fluxml.ai/Flux.jl/stable/models/overview/
using Flux
actual(x) = 4x + 2
x_train, x_test= hcat(0:5...), hcat(6:10...)
y_train, y_test = actual.(x_train), actual.(...
2
votes
1
answer
219
views
Linear combinations of Zygote.Grads
I am building and training a neural network model with Flux, and I am wondering if there is a way to take linear combinations of Zygote.Grads types.
Here is a minimalistic example. This is how it is ...
2
votes
2
answers
420
views
Custom Loss Function in Flux.jl
I am trying to implement a model with a custom loss function in the Flux.jl package. I include the code for a simplified model, but the error stays the same.
I have an interpolator which takes a ...
0
votes
1
answer
180
views
No decrease in loss after lots traning
I was trying to train a convolution network. But it is not improving, i.e. loss is not decreasing. And the train function is also terminating much more quickly than usual. Below is the minimal code to ...
5
votes
2
answers
236
views
How to know which module exports a certain function
I was going through this Flux.jl tutorial and came across something called Chain.
m = Chain(Dense(10, 5, relu), Dense(5, 2), softmax)
It was not imported from any of the used modules and no namespace ...
0
votes
1
answer
618
views
Flux.jl: How to sample random batch in dataloader?
I'm trying to randomly sample a mini-batch from Flux.dataloader rather than iterate through... I already created train_dataloader object, and all I need is random sampling from this object. Can you ...
3
votes
2
answers
535
views
Flux.jl : Customizing optimizer
I'm trying to implement a gradient-free optimizer function to train convolutional neural networks with Julia using Flux.jl. The reference paper is this: https://arxiv.org/abs/2005.05955. This paper ...
1
vote
0
answers
227
views
DimensionMisMatch Error Just Before Training
I’m trying to train a MLP model to predict number of real roots of polynomials. x_train and y_train include array of arrays such as [[-204, 20, 13, 1, 0]] which are coefficients of polynomials. x_test ...
7
votes
0
answers
685
views
I have a problem precompile the Flux package in Julia
I am trying to add Flux into my package repository in Julia but I am getting an error.
I typed pkg> add Flux but Julia REPL returns "5 dependencies errored".
And they are:
LLVM
...
1
vote
0
answers
69
views
why in julia Tracker.update!(W,-alpha*gW) and W = W - alpha * gW behaved differently?
I am very new to julia. While following tutorial in https://www.math.purdue.edu/~allen450/ML-with-Julia-Tutorial.html,
Tracker.update!(W,-alpha*gW)
and
W = W - alpha * gW
behaved differently. With W ...
1
vote
1
answer
199
views
LoadError: MethodError while training LSTM model in Julia
I'm trying to train an LSTM model to predict number of real roots of polynomials. x_train and y_train include array of arrays such as [[-204, 20, 13, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
2
votes
1
answer
394
views
Unexpected difference in behaviour with/out let block
I have been using Flux.jl and have been confused by the differences when running code within a let block and without. The following example runs without error:
using Flux
p = rand(2)
function f(x)
...
2
votes
0
answers
192
views
An unexpected variable reference changing during Julia Zygote gradient pullback
I'm using Julia Flux.jl to train models. But when I customize my model, there is an issue the variable reference in gradient pullback function doesn't behave as expected. For simplicity, the issue ...
2
votes
1
answer
407
views
How can I evaluate and take the derivative of a neural net in Julia
I have solved a differential equation with a neural net. I leave code below with an example. I want to be able to compute the first derivative of this neural net with respect to its input "x"...
1
vote
1
answer
321
views
Loading a Flux model into a MLJ machine
I would like to find out if it's possible to load a saved Flux model or its weights (a .bson file) into an MLJ machine. I would like to evaluate the performance of my model, using the Performance ...
1
vote
2
answers
538
views
Do you have to write a custom training loop to get the loss value in each epoch with Flux.jl?
Flux.jl provides a helpful train! function which when paired with the @epoch macro, can serve as the main training loop. However, unlike most custom training loops, there is no output info as to the ...
1
vote
1
answer
397
views
How to load an image for inference in Flux.jl?
I have a model which I trained using a specific dataset. I did not originally break the set up into a train and test set (which I should have). With that said, I want to do some adhoc testing to see ...
1
vote
1
answer
361
views
Flux.jl model always outputs 1.0 after adding Sigmoid activation function
My original issue was that I wanted my model to only output 0-1 so I can map back to my categorical images labels (Flux.jl restrict variables between 0 and 1). So I decided to add a sigmoid activation ...
3
votes
3
answers
678
views
How to check the input dimensions of a model in Flux.jl?
I have a resnet model which I am working with. I originally trained the model using batches of images. Now that it is trained, I want to do inference on a single image (224x224 with 3 color channels). ...
0
votes
1
answer
176
views
Load an arbitrary .BSON file into Flux.jl which wasn't created using Flux
Right now, I am trying to load an arbitrary .BSON model into Flux which looks like the following:
weights = ONNX.load_weights("weights.bson")
Dict{String, Any} with 521 entries:
"...
2
votes
0
answers
264
views
Integrodifferential PDE with NeuralPDEs - Julia
I am trying to numerically solve an integrodifferential PDE with NeuralPDE.jl. The equation has an analytical solution in the Laplace domain, and therefore I can validate the numerical solution. The ...
1
vote
0
answers
231
views
Training NN with Julia's Flux - Loss function with derivative of output and functions of output
I want to run this NN in which input is time over some interval. There's no label, and the loss function requires the derivative of the outputs and a specified function (H in my code), which is also a ...
3
votes
1
answer
733
views
How to use a .pth model in Flux.jl?
I have a model trained in PyTorch, saved in .pth format. Is it possible to use and load that model in Flux.jl? I looked around but did not see this mentioned anywhere in the Flux docs.
2
votes
1
answer
331
views
Multi label classification in Flux.jl?
I am currently working with a dataset in which the boundary between classes is not very well defined. I don't want to use regular classification since the nuance overlap of these classes might not be ...
2
votes
1
answer
382
views
How to freeze layer parameters in Flux.jl
I am currently working on a transfer learning problem so I would like to freeze most of my layers such that the model is not re-trained, and only the final layer's weights are modified. How can I ...
2
votes
0
answers
492
views
How to make use LSTM's in Flux.jl?
I am currently working with some Video data so rather than use a standard feed-forward model, I want to use a Long short-term memory (LSTM) architecture such that the context/memory persists in the ...
0
votes
0
answers
467
views
How to implement dropout in Flux.jl?
My current model is suffering from overfitting. I want to be able to add dropout to my model such that I prevent overfitting by shutting down some neurons. I was reading through this article on ...