Neural networks and the derivatives of the cost function

AI Thread Summary
The discussion centers on the challenges of deriving the derivatives of the quadratic cost function in artificial neural networks, particularly in relation to the weight matrices. The user expresses difficulty in determining when to apply the Hadamard product versus the dot product, as well as the correct order of operations in matrix multiplication. A suggestion is made to refer to a derivation found on stats.stackexchange, which emphasizes the importance of the chain rule for derivatives in this context. The conversation highlights the need for clear resources that explain these mathematical concepts in relation to neural networks.
2sin54
Messages
109
Reaction score
1
Hello. I need some guidance on the derivation of the derivatives of the quadratic cost function (CF) in an artificial neural network. I can derive the equations for the forward propagation with no trouble but when it comes to finding the derivative of the CF with respect to the weight matrix (matrices) I struggle to distinguish where to use the Hadamar product, where to use the dot matrix product and the order of the multiples. Does anyone know some good resources where I could see a thorough derivation of this OR linear algebra resource relevant to my question?
 
Technology news on Phys.org
Thread 'Star maps using Blender'

Thread 'Star maps using Blender'

Blender just recently dropped a new version, 4.5(with 5.0 on the horizon), and within it was a new feature for which I immediately thought of a use for. The new feature was a .csv importer for Geometry nodes. Geometry nodes are a method of modelling that uses a node tree to create 3D models which offers more flexibility than straight modeling does. The .csv importer node allows you to bring in a .csv file and use the data in it to control aspects of your model. So for example, if you...
View full post »

Thread 'Who is responsible for the software when AI takes over programming?'

I tried a web search "the loss of programming ", and found an article saying that all aspects of writing, developing, and testing software programs will one day all be handled through artificial intelligence. One must wonder then, who is responsible. WHO is responsible for any problems, bugs, deficiencies, or whatever malfunctions which the programs make their users endure? Things may work wrong however the "wrong" happens. AI needs to fix the problems for the users. Any way to...
View full post »

Thread 'Could the reason why I can't select any kernels in VS Code be this error?'

I am trying to run an .ipynb file and have installed Miniconda as well as created an environment as such -conda create -n <env_name> python=3.7 ipykernel jupyter I am assuming this is successful as I can activate this environment via the anaconda prompt and following command -conda activate <env_name> Then I downloaded and installed VS code and I am trying to edit an .ipynb file. I want to select a kernel, via VS Code but when I press the button on the upper right corner I am greeted...
View full post »

Similar threads

Normalization condition with a neural network
Replies
3
Views
1K
Can CNNs Be Tuned to Enhance Specific Texture Parameters in Image Simulation?
Replies
1
Views
1K
Derivative of Cost Function with Respect to Output Layer Weight
Replies
31
Views
3K
Neural network problems with multi-input dependencies
Replies
1
Views
2K
Application of Artificial Neural Networks to Business Problems
Replies
1
Views
2K
Programs Diploma thesis - Neural Network Application in Physical Problems
Replies
1
Views
2K
Can an ANN Use a SR-Latch to Store Fuzzy Floating Point Variables?
Replies
6
Views
2K
MHB Convexivity issues in a new type of network - NLP
Replies
1
Views
3K
How Can Artificial Neural Networks Enhance Radar Image Classification?
Replies
1
Views
3K
I Is GRW theory an interpretation of quantum mechanics or a rival theory?
4
Replies
169
Views
10K

Hot Threads

Recent Insights

Back
Top