Tuesday, July 22, 2014

A Gentle Introduction to Backpropagation



Why is this blog being written?

[Update: A more detailed version of this article with coding outline is available. Please read the note at the end of this article.]

Neural networks have always fascinated me ever since I became aware of them in the 1990s. They are often represented with a hypnotizing array of connections. In the last decade, deep neural networks have dominated pattern recognition, often replacing other algorithms for applications like computer vision and voice recognition. At least in specialized tasks, they indeed come close to mimicking the miraculous feats of cognition our brains are capable of.

While neural networks are capable of such feats, the very discovery of a method of programming such a computational device seems to me to be a miraculous feat of cognition worthy of celebration. My purpose in writing this blog is to share my perspective on an amazing algorithm made widely known by a 1986 publication in Nature.
I will assume that the reader has some understanding of the meaning and purpose of the various elements of a neural network such as the one shown in Figure 1. I have provided a little bit of background below. Other than that, this blog should be an easy read for those with some familiarity of basic calculus.

Figure 1: An example of a multi-layered neural network that can be used to associate an input consisting of 10 numbers with one of 4 decisions or predictions.

What is so difficult about designing a neural network?

To appreciate the difficulty involved in designing a neural network, consider this: The neural network shown in Figure 1 can be used to associate an input consisting of 10 numbers with one of 4 decisions or predictions. For example, the neural network shown may be used by a bank to determine if credit should be extended to a customer. In this case, the 10 input numbers represent various parameters relevant to an individual's financial responsibility such as balance in savings accounts, outstanding loan amounts, number of years of employment, and so on. The neural network takes in these 10 numbers, performs calculations and produces an output consisting of 4 numbers. Depending on the position at which the maximum of these 4 numbers appears in the output, the prediction could be one of the following:
  1. Excellent creditworthiness with high spending limit
  2. Average creditworthiness with moderate spending limit
  3. Low creditworthiness with low spending limit
  4. High default risk.
Based on this prediction, the bank would take the appropriate decision.

A neural network such as the one shown in Figure 1 can perform this miraculous feat of cognition only if it is specifically trained to do so. For the network to work correctly, the weight at each of its (10$\times $4)+(4$\times $6)+(6$\times $8)+(8$\times $4) =144 connections have to be carefully chosen such that the network classifies every input drawn from a known training set with a high degree of accuracy. This is a classic application of the supervised learning paradigm in machine learning.

There are, of course, no formulas in existence to directly set the values of the 144 weights. The only recourse is to start with some initial values for the 144 weights, check how good the resulting neural network is, and repeatedly refine the weights to progressively make the network more and more accurate. So, what is needed is a method of refining the weights.

If one were to think of the accuracy as some grand function of the weights, it makes sense to refine each weight by changing it by an amount proportional to the partial derivative of that grand function with respect to that weight. Why bring in partial derivatives? Because they precisely predict how the accuracy responds to small changes in the weights. In fact, at every iteration, performing a refinement guided by the partial derivatives results in a more advantageous gain in accuracy compared to any other method of refinement. The method of steepest descent does exactly what is suggested here – with the minor, but entirely equivalent, goal of seeking to progressively decrease the error, rather than increase the accuracy.

To keep things straight, let me list the concepts I have described thus far:
  1. The weights of the neural network must be set such that the error on a known training set is minimized. Ideally, the network must yield zero error on a large and representative training data set.
  2. For any given setting of weights, we can attempt to refine them by changing each weight by an amount proportional to the partial derivative of the error with respect to that weight. The partial derivatives themselves change after any such refinement, and must be recomputed.
  3. We can keep on refining the weights in this manner, and stop refining when the error is zero. In real life, we call it done when the error is low enough. Or refuses to fall anymore. Or we run out of time after a few million rounds of refinements.
Seems simple, right? Yes. As long as there is a simple way of calculating the partial derivative of error with respect to every weight. In principle, the partial derivatives can be calculated by systematically perturbing the weights and measuring the resulting changes in error. A word to the wise, don't try this at home. Or even on your neighborhood supercomputer.


Backpropagation

In fact, this error minimization problem that must be solved to train a neural network eluded a practical solution for decades till D. E. Rumelhart, C. E. Hinton, and R. J. Williams (drawing inspiration from other researchers) demonstrated a technique, which they called backpropagation, and made it widely known (Nature 323, 533-536, 9 October 1986). It is essentially by building upon their method that today others have ventured to program neural networks with 60 million weights, with astounding results.

According to Bernard Widrow, now Professor Emeritus at Stanford University and one of the pioneers of neural networks "The basic concepts of backpropagation are easily grasped. Unfortunately, these simple ideas are often obscured by relatively intricate notation, so formal derivations of the backpropagation rule are often tedious." This is indeed unfortunate because the backpropagation rule is one of the most elegant applications of calculus that I have known. 

Easy as 1-2-3
Once you appreciate the fact that, in order to train a neural network, you need to somehow calculate the partial derivatives of the error with respect to weights, backpropagation can be easily and qualitatively derived by reducing it to three core concepts. It also helps immensely to keep the notation intuitive and easy to connect to the concept being symbolized.
Figure 2: The derivation of the backpropagation algorithm is simplified by adding an extra computational block to calculate the error and also by boxing parts of the network.

1. Boxing

Since training the neural network is all about minimizing the training error, the first step in the derivation involves tacking on an extra computational block to calculate the error between the actual output $\left\{o_1,o_2,o_3,o_4\right\}$ and a known target $\{t_1,t_2,t_3,t_4\}$. This is shown as a triangular block in Figure 2. For now, let us think of the output and the target as known and fixed entities. Although we need not concern ourselves with the exact formula to compute the error, I offer the familiar sum-of-squares error as an example
\[\text{e=(}o_1-t_1)^2\text{+(}o_2-t_2)^2\text{+(}o_3-t_3)^2\text{+(}o_4-t_4)^2\]

Next, we choose one of the layers (say Layer 3) and enclose that layer and all following layers (including the error calculating block) in a box, as shown in gray in Figure 2. Keep in mind that this is just one of several nested boxes we can construct in order to compartmentalize the network. Let us resolve not to worry about anything going on inside the box but simply think of the relationship between the input to the box and the output (i.e the error) coming out of the box. We will call this box the current box, and call the input to this box the current input$\{c_1,c_2,c_3,c_4,c_5,c_6\}$. It is important to recognize that the functional relationship between the current input to the box and the output emanating from the box is completely defined and can be computed. Let us denote this function as $E(c_1,c_2,c_3,c_4,c_5,c_6)$.
Figure 3: The box contains parts of the neural network hidden from view. It allows us to think about broad relationships among different parts of the network.

2. Sensitivity

As our journey through backpropagation continues, I gently request you to assume that the vector of partial derivatives $\frac{\partial E}{\partial c_1},\frac{\partial E}{\partial c_2}, \ldots
$ of the function $E(c_1,c_2,c_3,c_4,c_5,c_6)$ is known. This might seem like a stretch. After all, we have set out to find a method to compute some other (equally unrealized) partial derivatives. But I assure you it will all work out in the end. To emphasize the crucial nature of this simple concept, it has been given a name: Sensitivity. Let us denote the sensitivity of our current box as $\{\text{$\delta $c}_1,\text{$\delta $c}_2,\text{$\delta $c}_3,\text{$\delta
   $c}_4,\text{$\delta $c}_5,\text{$\delta $c}_6\}$. With the help of Figure 3 to hold these concepts in our mind, we can concretely think about how the output of the current box responds to a small perturbation applied to any of its current inputs. For example, if the fourth component of the current input changes by the small amount $\Delta c_4,$ we can expect the error at the output to change by $\Delta c_4 \delta c_4.$ Further, in addition to the hypothetical change in component 4, if there is a simultaneous change of $\Delta c_6$ in component 6, we can expect the error at the output to change by an additional amount, making the total change $\Delta c_4 \delta c_4 + \Delta c_6 \delta c_6.$ The effect of small simultaneous changes in the current input components simply add up at the output.

Knowing the sensitivity of the current box, what can we say about the sensitivity of the preceding box? Keep in mind that the preceding box encloses Layer 2 and all following layers, including the error calculating block. For our specific example, let us call the input to this box the preceding input, $\{p_1,p_2,p_3,p_4.\}$ It follows quite logically that the sensitivity of the preceding box (which we will naturally denote as $\{\text{$\delta $p}_1,\text{$\delta $p}_2,\text{$\delta $p}_3,\text{$\delta
   $p}_4\}$) must be related to the sensitivity of the current box and the extra neural network elements making up the difference between the two nested boxes. The extra elements are the very vital nonlinear activation function units, summing junctions and weights.

Figure 4(a) shows the current box and the extra elements that must be added to construct the preceding box. For clarity, all the elements not relevant to the calculation of the first component of sensitivity ($\delta p_1$) have been grayed out. Look closely at Figures 4(a) and 4(b) to understand how the sensitivity of the preceding box can easily be derived from first principles. Specifically, Figure 4(b) provides insight into how $\delta p_1(=\frac{\partial e}{\partial p_1})$ can be computed  by allowing the input component $p_1$ to change by a small quantity $\Delta p_1$ and following the resulting changes in the network. Notes: (i) The notation $\mathcal{A}'\left(p_1\right)$ has been used for the derivative of the activation function evaluated at $p_1.$ (ii) For clarity, not all changes in signals have been explicitly labeled. Those that are not labeled can easily be determined since they all follow an obvious pattern.
Figure 4(a) shows the current box and the extra elements that must be added to construct the preceding box. Figure 4(b) provides insight into how $\delta p_1(=\frac{\partial e}{\partial p_1})$ can be computed  by allowing the input component $p_1$ to change by a small quantity $\Delta p_1$ and following the resulting changes in the network.
This is indeed a deep result. (And one which we have managed to arrive at without recourse to tedious calculus and numbered equations.) By repeated application, this result allows us to work backwards and calculate the sensitivity of the error to changes in the input at every layer.

The algorithm gets the name backpropagation because the sensitivities are propagated backwards as they are calculated in sequence. The textbook formula to express the sensitivity of the preceding layer in terms of the sensitivity of the current layer is easily seen to be
\[\delta p_i = \mathcal{A}'(p_i) \sum _{j} w_{i j}\delta c_j \]

A starting point is all we need to completely calculate all the sensitivity terms throughout the neural network. To do this, we consider the error computing block itself as the first box. For this box, the input is $\left\{o_1,o_2,o_3,o_4\right\},$ and the output is $e$ as given in the sum-of-squares error formula we have seen before. Simple calculus gives us the components of the sensitivity of the error computing block
\[\left\{
2 \left(o_1-t_1\right),
2 \left(o_2-t_2\right),
2 \left(o_3-t_3\right),
2 \left(o_4-t_4\right)
\right\}\]

3. Weight updates

At this point, the last section writes itself. Following the same strategy outlined in the previous figure, look at Figure 5(a) and 5(b) to intuitively understand how the error changes in response to a small change in one of the weights, say $w_{1 1}$. Once again in these figures, details of connections not immediately relevant to this calculation have been grayed out. The much sought after partial derivative of error with respect to the specific weight $w_{1 1}$ is easily seen to be $\mathcal{A}\left(p_1\right) \delta c_1$. The textbook formula to compute the partial derivative of the error with respect to any weight is easily seen to be
\[\frac{\partial e}{\partial w_{i j}} =\mathcal{A}\left(p_i\right) \delta c_j \]
Figure 5: Another set of figures designed to provide insight into the error changes in response to a small change in one of the weights, $w_{1 1}$. Using this intuition, the partial derivative $\frac{\partial e}{\partial w_{1 1}}$ can be computed as $\mathcal{A}\left(p_1\right) \delta c_1$.
Now that we have a formula to calculate the partial derivative of error with respect to every weight in the network, we can proceed to iteratively refine the weights and minimize the error using the method of steepest descent.

In the most popular version of backpropagation, called stochastic backpropagation, the weights are initially set to small random values and the training set is randomly polled to pick out a single input-target pair. The input is passed through the network to compute internal signals (like $\mathcal{A}\left(p_1\right)$ and $\mathcal{A}'\left(p_1\right)$ shown in Figures 4 and 5) and the output vector. Once this is done, all the information needed to initiate backpropagation becomes available. The partial derivatives of error with respect to the weights can be computed, and the weights can be refined with intent to reduce the error. The process is iterated using another randomly chosen input-target pair.

The miraculous feat of cognition

I am in awe of the miraculous feat of cognition that lead early neural network researchers to arrive at the backpropagation algorithm. They clearly had the ability to see patterns and make elegant groupings which ultimately made it possible to train huge networks. Their work not only resulted in the neural network applications we use today, but have also inspired a host of other related algorithms which depend on error minimization.
Although this algorithm has been presented here as a single established method, it should be regarded as a framework. In my experience, appreciating how an algorithm is derived leads to insight which makes it possible to explore beneficial variations. The process of designing robust and efficient scientific algorithms frequently leads us to regard established frameworks as substrates upon which to build new and better algorithms.


Note: A PDF version of this article with high quality graphics and pseudocode ("Training Wheels for Training Neural Networks") for implementing the algorithm is available. Please visit www.numericinsight.com and find Downloads on the home page.

173 comments:

  1. Good and easy grasp explanation. Best :)

    ReplyDelete
  2. The best explanation I've seen on this topic yet!

    ReplyDelete
  3. Great explanation. However I would recommend not having animations but instead just images with the red/yellow boxes added. It is a bit frustrating to read them and then wait until they come back.

    ReplyDelete
  4. I worked on a classic problem of using a neural network to play the Japanese game GO. Training a
    NN on a 19x19 board is to computationally expensive. Humans learn on a 9x9 board. So the question
    is, can we train a NN on a 9x9 board and apply the learning to the full board? The solution is to think
    about the 9x9 NN as a filter that can "slide around" the 19x19, looking for a match. A fast fourier
    transform can be used to do this. The FFT can "look for" the 9x9 pattern over the 19x19 board.
    So you can "teach in the small" and "learn in the large".

    I never wrote up and published this result and I haven't seen it anywhere else to date.
    It is useful for complex problems (e.g. finding enemy tanks in a forest) involving large data
    sets (high density images).

    ReplyDelete
  5. Can I repost this blog and translate it into Chinese by my understanding ? It is very helpfully to understand backpropagation.

    ReplyDelete
    Replies
    1. Thank you for your interest. Please email me at shashi@numericinsight.com to discuss this matter.

      Delete
  6. Sir how can the same algorithm be used for Texture analysis

    ReplyDelete
  7. thanks for sharing wonderful article description of what while written the blogs in back end process. your graphical information is too good. keep updating more.
    Web Designing Training in chennai

    ReplyDelete
  8. Thank you for taking the time and sharing this information with us. It was indeed very helpful and insightful while being straight forward and to the point.
    mcdonaldsgutscheine | startlr.com/ | saludlimpia

    ReplyDelete
  9. Thank you for taking the time and sharing this information with us. It was indeed very helpful and insightful while being straight forward and to the point.
    PHP Training Institute in Chennai

    ReplyDelete
  10. It would have been the happiest moment for you,I mean if we have been waiting for something to happen and when it happens we forgot all hardwork and wait for getting that happened.

    python interview questions and answers
    python tutorials
    python course institute in electronic city

    ReplyDelete
  11. Interesting post which makes you think about a lot of things. This post helped to make the career.
    datapower tutorial video

    ReplyDelete
  12. Very nice post here thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    machine learning training in chennai
    machine learning training in omr
    top institutes for machine learning in chennai
    Android training in chennai
    PMP training in chennai

    ReplyDelete
  13. Really great post, Thank you for sharing This knowledge.Excellently written article, if only all bloggers offered the same level of content as you, the internet would be a much better place. Please keep it up!
    angularjs Training in chennai

    angularjs Training in chennai

    angularjs-Training in tambaram

    angularjs-Training in sholinganallur

    angularjs-Training in velachery

    angularjs-Training in pune

    ReplyDelete
  14. I really like the dear information you offer in your articles. I’m able to bookmark your site and show the kids check out up here generally. Im fairly positive theyre likely to be informed a great deal of new stuff here than anyone
    Data Science Training in Chennai
    Data Science training in kalyan nagar
    Data science training in Bangalore
    Data Science training in marathahalli
    Data Science interview questions and answers
    Data science training in jaya nagar
    Data science training in bangalore

    ReplyDelete
  15. This is really impressive post. I would like to appreciate you for making it very simple and easy.
    Regards,
    Best Devops Training in Chennai | Best Devops Training Institute in Chennai

    ReplyDelete
  16. Thanks for posting useful information.You have provided an nice article, Thank you very much for this one. And i hope this will be useful for many people.. and i am waiting for your next post keep on updating these kinds of knowledgeable things...Really it was an awesome article...very interesting to read..please sharing like this information......
    mi service center in chennai
    redmi service center in chennai
    xiaomi service centre chennai
    redmi service center
    mi service center
    redmi service center near me
    redmi mobile service centre in chennai

    ReplyDelete
  17. Really nice post.provided a helpful information.I hope that you will post more updates like this
    AWS Online Training

    ReplyDelete

  18. Great Post,really it was very helpful for us.
    Thanks a lot for sharing!
    I found this blog to be very useful!!
    Hadoop training in Bangalore

    ReplyDelete
  19. It’s interesting content and Great work. Definitely, it will be helpful for others. I would like to follow your blog. Keep post

    Check out:
    react js course fees
    react interview questions medium
    reactjs interview questions pdf

    ReplyDelete
  20. Si el agua cae al lago, desaparecerá( phụ kiện tủ bếp ). Pero si cae a la hoja de( phụ kiện tủ áo ) loto, brillará como una joya. Caer igual pero( thùng gạo thông minh ) estar con alguien es importante.

    ReplyDelete
  21. Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading Python classes in pune new articles. Keep up the good work!

    ReplyDelete
  22. Memilih meja yang cocok untuk anda
    Setiap meja memiliki level masing-masing dari nominal paling kecil sampai besar dan tingkat dari pemula sampai master.
    asikqq
    dewaqq
    sumoqq
    interqq
    pionpoker
    bandar ceme terbaik
    hobiqq
    paito warna
    forum prediksi

    ReplyDelete
  23. VERY NICE POST ....THANKS FOR SHARING
    https://www.manishafashion.com/how-to-lose-weight-in-one-month/how-to-lose-weight-in-one-month-manishafashion-com/

    ReplyDelete
  24. This comment has been removed by the author.

    ReplyDelete
  25. Epson Printer Support Phone Number
    has turned into the need of each individual utilizing innovation related howdy tech items. PC and printer clients additionally experience numerous issues at the season of utilization which propel them to secure the client support number(+1-888-326-0222) and get their help to determine the issues. As the innovation of a printer and comparable gadgets are not easy to use for some clients, subsequently, Epson Printer Support Phone Number is accessible online to help end-clients to tackle tech related issues.

    ReplyDelete
  26. I love to read your articles because your writing style is too good, This article is really very interesting and effective.So keep sharing post like this. ExcelR Solutions

    ReplyDelete
  27. Data Science AcadmyAugust 9, 2019 at 2:56 AM

    Business Analytics or Data Analytics or data science training in hyderabad is an extremely popular, in-demand profession which requires a professional to possess sound knowledge of analysing data in all dimensions and uncover the unseen truth coupled with logic and domain knowledge to impact the top-line (increase business) and bottom-line (increase revenue).

    ReplyDelete
  28. Nice post thank you for sharing. Printer Tech Support Number 1-800-289-8149 for help you contact.Call us We are 24*7 available. https://www.printertechssupportnumber.com/

    ReplyDelete
  29. Nice post thank you for sharing. Antivirus Tech Support Number 1-800-524-2156 for your help.Call us We are 24*7 available. https://www.antivirustechsupports.com/avast-support/

    ReplyDelete
  30. Norton.com/nu16 may be very actually only a well-known trusted brand over the world of online security software applications solutions and providing high-quality anti-virus security software with latest and improved features.if you have any issues related to then contact us Norton.com/setup

    ReplyDelete
  31. Resources like the one you mentioned here will be very useful to me! I will post to this page on my blog. I am sure my visitors will find that very useful...
    Thanks and Regards,
    Digital Marketing Course

    ReplyDelete
  32. Office.com/setup but all the software for all time has the setup file, and it is very significant, without the setup file the software turn ineffective.Know how to benefit,

    download, install, set in movement, uninstall and reinstall MS office setup and Get Started by now Office setup. if you have any query related to officecom then contact us.
    www.office.com/setup |www.norton.com/setup

    ReplyDelete
  33. Microsoft Office PC Setup Office Remote transforms your telephone into a brilliant remote that connects with Microsoft Office on your PC. The application
    gives you a chance to control Word, Exceed expectations, and PowerPoint, Outlook from over the room, so you can stroll around uninhibitedly during introductions.


    Need Help Click Here. www.office.com/setup


    www.office.com/setup

    Get started Norton Setup with Product Key at norton.com/setup or call us toll-free number for instant support.

    www.norton.com/setup

    ReplyDelete
  34. For Devops training in Bangalore Visit:
    Devops Training in Bangalore

    ReplyDelete
  35. I am Grateful™ that you shared this informational post. ✆ Your website has everything that I have been looking for so long. Your knowledge about this topic is quite impressive. I am amazed by the content you shared on this website.☞ I loved this page so much that I have bookmarked it for future reference. Thank you for this valuable piece of information. Sadly I could not find any additional information on this page. The information available here is similar to other websites. I like that your understanding about this subject is very deep. Visit௹☞ Telstra WebmailPlumbing companies near mewww.norton.com/setupoffice setupwww.office.com/setupnorton nu16 install

    ReplyDelete
  36. Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area.

    Best PHP Training Institute in Chennai|PHP Course in chennai
    Best .Net Training Institute in Chennai
    Big Data Hadoop Training in Chennai
    Linux Training in Chennai
    Cloud Computing Training in Chennai

    ReplyDelete
  37. Excellent information with unique content and it is very useful to know about the python.python training in bangalore

    ReplyDelete
  38. This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information.devops training in bangalore

    ReplyDelete
  39. I really enjoy reading this article.Hope that you would do great in upcoming time.A perfect post.Thanks for microsoft azure sharing.microsoft azure training in bangalore

    ReplyDelete
  40. Congratulations This is the great things. Thanks to giving the time to share such a nice information.best Mulesoft training in bangalore

    ReplyDelete
  41. Excellent post for the people who really need information for this technology.ServiceNow training in bangalore

    ReplyDelete
  42. Very useful and information content has been shared out here, Thanks for sharing it.Mulesoft training in bangalore

    ReplyDelete
  43. Awesome post with lots of data and I have bookmarked this page for my reference. Share more ideas frequently.data science training in bangalore

    ReplyDelete
  44. Awesome post with lots of data and I have bookmarked this page for my reference. Share more ideas frequently.oracle apps scm training in bangalore

    ReplyDelete
  45. This post is very simple to read and appreciate without leaving any details out. Great work!

    Please check ExcelR Data Science Course Pune

    ReplyDelete
  46. Thanks for one marvelous posting! I enjoyed reading it; you are a great author. I will make sure to bookmark your blog and may come back someday. I want to encourage that you continue your great posts.dot net training in bangalore

    ReplyDelete
  47. You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!!
    ExcelR Data Analytics Course

    ReplyDelete
  48. I read this post your post so nice and very informative post thanks for sharing this post.

    Real Time Experts provides Best SAP PM Training in Bangalore with expert real-time trainers who are working Professionals with min 8+ years of experience in Java Training Industry, we also provide 100% Placement Assistance with Live Projects on Java Training.

    ReplyDelete
  49. "Just saying thanks will not just be sufficient, for the fantastic lucidity in your writing. I will instantly grab your articles to get deeper into the topic. And as the same way ExcelR also helps organisations by providing data science courses based on practical knowledge and theoretical concepts. It offers the best value in training services combined with the support of our creative staff to provide meaningful solution that suits your learning needs.

    Business Analytics Courses "

    ReplyDelete
  50. I am happy for sharing on this blog its awesome blog I really impressed. thanks for sharing. Great efforts.

    Start your journey with AWS Course and get hands-on Experience with 100% Placement assistance from Expert Trainers with 8+ Years of experience @eTechno Soft Solutions Located in BTM Layout Bangalore.

    ReplyDelete
  51. A great website with interesting and unique material what else would you need.

    360digitmg IOT Training Courses

    ReplyDelete
  52. I can’t imagine that’s a great post. Thanks for sharing.

    Start your journey with In Software Training in Bangalore and get hands-on Experience with 100% Placement assistance from experts Trainers @eTechno Soft Solutions Located in BTM Layout Bangalore.

    ReplyDelete
  53. Really a awesome blog for the freshers. Thanks for posting the information.
    Machine Learning Training in Delhi

    ReplyDelete
  54. This is a blog to watch for sure. I think you have a really awesome writing style by the way. Very easy to read. Your blog design is so clean too! Thank you all the hard work!
    AngularJS Training in Pune
    RPA Training in Pune
    Devops Training in Pune

    ReplyDelete
  55. Snapdeal Winner List here came up with an Offer where you can win Snapdeal prize list 2020 by just playing a game & win prizes.
    Snapdeal winner name2020 also check the Snapdeal lucky draw2020

    ReplyDelete


  56. I am really happy to say it’s an interesting post to read . I learn new information from your article , you are doing a great job . Keep it up and a i also want to share some information regarding selenium training course and selenium videos

    ReplyDelete
  57. Microsoft Office arrangement is a finished bundle of Microsoft that incorporates Microsoft Word, Excel, Access, PowerPoint, and Outlook. To get support for Download, Install MS Office on your PC in this way, you can tap on the accompanying connection.
    HOW TO FIX MICROSOFT OFFICE NOT ACTIVATED ERROR?

    ReplyDelete
  58. Attend The Machine Learning courses in Bangalore From ExcelR. Practical Machine Learning courses in Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Machine Learning courses in Bangalore.
    ExcelR Machine Learning courses in Bangalore

    ReplyDelete
  59. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
    data analytics courses

    data science interview questions

    business analytics course

    data science course in mumbai

    ReplyDelete
  60. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    top angularjs online training

    ReplyDelete
  61. When I originally commented I seem to have clicked the -Notify me when new comments are added- checkbox and from now on each time a comment is added I recieve 4 emails with the exact same comment. Perhaps there is an easy method you are able to remove me from that service? Thanks a lot!
    Tech PC

    ReplyDelete
  62. I am inspired with your post writing style & how continuously you describe this topic. After aws training videos reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.aws videos

    ReplyDelete
  63. Study Machine Learning Course Bangalore with ExcelR where you get a great experience and better knowledge .
    Machine Learning Course Bangalore

    ReplyDelete
  64. Study Machine Learning Training in Bangalore with ExcelR where you get a great experience and better knowledge .
    Business Analytics course

    ReplyDelete
  65. I have to search sites with relevant information on given topic ExcelR Machine Learning Courses In Pune and provide them to teacher our opinion and the article.

    ReplyDelete
  66. Study Business Analytics Course in Bangalore with ExcelR where you get a great experience and better knowledge.
    Business Analytics Course

    ReplyDelete
  67. Great post! We are linking to this particularly great article on our website. lap Keep up the great writing.

    ReplyDelete
  68. Your blog is quite helpful to me and i am sure to others too. I appreciate your post, it is usually nice and contains useful information. Home elevators Melbourne
    Home lifts

    ReplyDelete
  69. Your blog is quite helpful to me and i am sure to others too. I appreciate your post, it is usually nice and contains useful information.
    Home elevators
    Home elevators Melbourne
    Home lifts

    ReplyDelete

  70. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.
    data analytics courses

    ReplyDelete
  71. Thanks for the informative article About Angularjs. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  72. This is exactly the information I'm looking for, I couldn't have asked for a simpler read with great tips like this... Thanks! ExcelR Data Science Course In Pune

    ReplyDelete
  73. Your style is really unique in comparison to other people I have read stuff from. Thanks for posting when you have the opportunity, Guess I will just bookmark this page.
    Jio Lottery Winner

    ReplyDelete
  74. I like viewing websites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you!

    Correlation vs Covariance

    ReplyDelete
  75. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    DevOps Training in Chennai

    DevOps Course in Chennai

    ReplyDelete
  76. Nice & Informative Blog !
    If you are looking for the best accounting software that can help you manage your business operations. call us at QuickBooks Phone Number 1-(855) 550-7546.

    ReplyDelete
  77. Thanks for sharing such useful information with us. I hope you will share some more info about your blog. Please Keep sharing. We will also provide Quickbooks Parsing Error Contact us +1-877-756-1077 for instant help.

    ReplyDelete
  78. Nice Article, Great experience for me by reading this info. Thanks for sharing the information with us.keep updating your ideas.

    Looking for the best PPC course in Bangalore India? Learn PPC from Ranjan Jena, 10+ Years Expert Google Ads Trainer. 1000+ Students Trained @ eMarket Education, Koramangala, Bangalore.

    Best Online Digital Marketing Course in Bangalore, India
    Best Digital Marketing Institute in Bangalore

    ReplyDelete
  79. This was not just great in fact this was really perfect your talent in writing was great.ExcelR Data Analytics Courses

    ReplyDelete
  80. Very interesting article to read it. I would like to thank you for the efforts you had made for writing this wonderful article. This article inspired me to read more. Keep sharing on updated posts…

    Learn Digital Marketing Course in Bangalore with Live Project Work & Case Studies taught by Ranjan Jena (10Yrs Trainer). 100% Guarantee to Clear Job Interview
    PPC Google Adwords Course
    Google Analytics Course
    Graphic Designing Course
    <a href="https://www.emarketeducation.in/digital-marketing-real-time-project-work”>Real-Time Project Work</a>

    ReplyDelete
  81. Nice & Informative Blog !
    Are you looking for extensive solutions for QuickBooks Error 12152? Do not worry at all. Just reach us via our QuickBooks Error Support Number and solve all your troubles related to QuickBooks in less time.

    ReplyDelete
  82. Hey there it awesome to read and explore your blog my friend. Keep creating like this! Thanks for sharing!
    Oracle Apps scm training in bangalore

    ReplyDelete
  83. Fiducia Solutions is an ISO 9001:2015 certified institute providing course certifications to all its students. We, at Fiducia Solutions, see to it that the candidate is certified and entitled to bag a good position in acclaimed companies. We provide certificates that are valued, and our alumni reputation proves that we are good at what we offer.

    And that is not all! We are known to provide 100% live project training and 100% written guaranteed placements to our students. That’s what makes us the best PHP/ HR/ Digital Marketing training institutes in Noida and Ghaziabad.

    PHP Training Institute in Noida
    HR Training Institute in Noida
    Digital Marketing Training Institute in Noida
    Android Training Institute in Noida

    ReplyDelete
  84. Nice Blog, i really want to read more about this topic, please keep posted regularly.
    If you face any Error in QuickBooks then immediately contactQuickbooks Error Support

    ReplyDelete
  85. Thanks for sharing good information it helps a lot for everyone. You can also check.
    python training in bangalore

    ReplyDelete
  86. Awesome,very useful for blog writers like me. Thanks for posting this visit us Python Courses Fees

    ReplyDelete
  87. https://www.achieversit.com/digital-marketing-training-course-in-bangalore

    ReplyDelete

  88. Hey! Mind-blowing blog. Keep writing such beautiful blogs. In case you are struggling with issues on QuickBooks software, dial QuickBooks Support Phone Number . The team, on the other end, will assist you with the best technical services.

    ReplyDelete
  89. AMAZING BACHELORETTE PARTY SUPPLIES We strive to have a positive impact on small to medium businesses, customers, employees, the economy, and communities. Surjmor bring together smart, passionate builders with different backgrounds and goals, who share a common desire to always be learning and inventing on behalf of our customers. With all the family of business that are a part of us, our goals is providing customers with the best service possible.

    xxxtoys.top

    ReplyDelete
  90. Amazing write-up! Really Good.
    Now the foundation of any business is marketing. Adsify marketing is the best Digital marketing in Trivandrum.

    ReplyDelete
  91. Hey! Nice Blog, I have been using QuickBooks for a long time. One day, I encountered QuickBooks Customer Service in my software, then I called QuickBooks Customer Service (855)963-5959. They resolved my error in the least possible time.

    ReplyDelete
  92. I am really impressed with your description and content. Thank You for sharing such a great blog. I hope you keep posting this type of informative post. First DigiAdd

    ReplyDelete
  93. Nice information, you write very nice articles, I visit your website for regular updates. There is great opportunity to making career in Airline Industry with help of Cabin Crew Training Centre

    ReplyDelete
  94. Superplasticizer Market Status (2016-2020) and Forecast (2021E-2028F) by Region, Product Type & End-Use
    Superplasticizer market

    Overview

    At the beginning of a recently published report on the global Superplasticizer market, extensive analysis of the industry has been done with an insightful explanation. The overview has explained the potential of the market and the role of key players that have been portrayed in the information that revealed the applications and manufacturing technology required for the growth of the global Superplasticizer market.

    Superplasticizer market

    ReplyDelete
  95. I would love to see your next update. Nice Post! Thank you.
    Henna Powder For Hair

    ReplyDelete
  96. Hello!!
    Nice Blog,Good content. We provide a best Quickbooks Customer Serviceyou can contact us at.+1 888-471-2380,PA.

    ReplyDelete
  97. I read blogs on a similar topic, but I never visited your blog. I added it to favorites and I’ll be your constant reader. Billionaire Boys Club Varsity Jacket

    ReplyDelete
  98. Nibav Vacuum Lifts one of the leading home lift companies in India, offers this unique technology in India. Vacuum Lifts are one of the most efficient home lifts, according to interior designers and architects who prescribe them most often.

    ReplyDelete
  99. Very useful tutorials and very easy to understand. Thanks for sharing. Ziyyara Edutech brings you the perfect solution with our comprehensive online English classes in Riyadh, Saudi Arabia.
    For more info visit English language classes in Riyadh or Call +971505593798

    ReplyDelete
  100. Thanks For Sharing.Nice Blog.keep posting more blogs. Ziyyara Edutech’s online tuition center offers personalized and comprehensive education, specifically designed to cater to the unique needs of Class 7 students.
    Book A Free Demo Today visit Home tuition classes for class 7

    ReplyDelete
  101. This article provides a beautifully clear and insightful introduction to the complex world of backpropagation in neural networks. The author's passion for the subject matter shines through, making it an engaging and informative read for both beginners and those with some familiarity in the field. Great work!
    Data Analytics Courses in Nashik

    ReplyDelete
  102. This article probably reflects on the purpose and motivations behind the blog, offering insights into the author's intentions and goals for their writing.

    Data Analytics Courses In Kochi



    ReplyDelete
  103. It is a powerful technique that has paved the way for many breakthroughs in machine learning and artificial intelligence. I appreciate he way of explanation in this post.
    Data Analytics Courses In Chennai

    ReplyDelete
  104. Neural networks are a cornerstone of artificial intelligence, mimicking the human brain's complex processing to solve intricate problems. In the realm of data analytics, Glasgow's Data Analytics courses offer a robust foundation for understanding and applying neural networks to extract valuable insights from data. Please also read Data Analytics courses in Glasgow .

    ReplyDelete
  105. The blog post introduces backpropagation in neural networks, emphasizing its role in minimizing errors and training the network. It simplifies the concept into three core ideas: boxing, sensitivity, and weight updates. The author highlights the algorithm's significance in the development of related deep learning and pattern recognition methods.
    Digital Marketing Courses In Spain

    ReplyDelete
  106. Thank you for providing in depth information and insightful knowledge on designing a neural network.
    Digital Marketing Courses In Bhutan

    ReplyDelete
  107. The inclusion of common challenges and potential pitfalls, along with tips on troubleshooting, added a practical dimension to the tutorial. It's evident that you aimed not only to explain the algorithm but also to equip readers with the knowledge to apply it effectively.

    Digital marketing for business

    ReplyDelete
  108. Congrats for nicely explaining the concepts. Well researched and thoughtful article.
    Investment banking courses after 12th

    ReplyDelete
  109. This comment has been removed by the author.

    ReplyDelete

  110. "Dive into the complex yet fascinating world of neural networks with the blog on 'A Gentle Introduction to Backpropagation.' This illuminating read is a compass for both beginners and enthusiasts, unraveling the mysteries of backpropagation with clarity and depth. Embark on a journey where the intricacies of learning algorithms are demystified, paving the way for a deeper understanding of the backbone of modern machine learning."
    Investment banking as a career in India

    ReplyDelete
  111. Ecola is on the forefront of high-tech, low-toxicity methods to combat termites, rodents and other pests without exposing your family and pets to toxic fumes.

    Our alternate treatments aren’t just good for the environment. They’re also good for your pocketbook. In most cases, Ecola’s non-chemical treatments are the most convenient and economical methods to rid your home or business of pesky pests.

    ReplyDelete

I welcome your comments. Do let me know if you have a numerical and insightful story to tell.