Saturday, June 6, 2020

Identify JS heap memory leaks with Allocation Timelines

Identify JS heap memory leaks with Allocation Timelines

The Allocation Timeline is another tool that can help you track down memory leaks in your JS heap.

To demonstrate the Allocation Timeline consider the following code:

var x = [];

function grow() {
  x
.push(new Array(1000000).join('x'));
}

document
.getElementById('grow').addEventListener('click', grow);

Every time that the button referenced in the code is pushed, a string of one million characters is added to the x array.

To record an Allocation Timeline, open DevTools, go to the Profiles panel, select the Record Allocation Timeline radio button, press the Start button, perform the action that you suspect is causing the memory leak, and then press the stop recording button (stop recording button) when you're done.

As you're recording, notice if any blue bars show up on the Allocation Timeline, like in the screenshot below.

new allocations

Those blue bars represent new memory allocations. Those new memory allocations are your candidates for memory leaks. You can zoom on a bar to filter the Constructor pane to only show objects that were allocated during the specified timeframe.

zoomed allocation timeline

Expand the object and click on its value to view more details about it in the Object pane. For example, in the screenshot below, by viewing the details of the object that was newly allocated, you'd be able to see that it was allocated to the x variable in the Window scope.

object details

Discover detached DOM tree memory leaks with Heap Snapshots

Discover detached DOM tree memory leaks with Heap Snapshots

A DOM node can only be garbage collected when there are no references to it from either the page's DOM tree or JavaScript code. A node is said to be "detached" when it's removed from the DOM tree but some JavaScript still references it. Detached DOM nodes are a common cause of memory leaks. This section teaches you how to use DevTools' heap profilers to identify detached nodes.

Here's a simple example of detached DOM nodes.

var detachedTree;

function create() {
 
var ul = document.createElement('ul');
 
for (var i = 0; i < 10; i++) {
   
var li = document.createElement('li');
    ul
.appendChild(li);
 
}
  detachedTree
= ul;
}

document
.getElementById('create').addEventListener('click', create);

Clicking the button referenced in the code creates a ul node with ten li children. These nodes are referenced by the code but do not exist in the DOM tree, so they're detached.

Heap snapshots are one way to identify detached nodes. As the name implies, heap snapshots show you how memory is distributed among your page's JS objects and DOM nodes at the point of time of the snapshot.

To create a snapshot, open DevTools and go to the Profiles panel, select the Take Heap Snapshot radio button, and then press the Take Snapshot button.

take heap snapshot

The snapshot may take some time to process and load. Once it's finished, select it from the lefthand panel (named HEAP SNAPSHOTS).

Type Detached in the Class filter textbox to search for detached DOM trees.

filtering for detached nodes

Expand the carats to investigate a detached tree.

investigating detached tree

Nodes highlighted yellow have direct references to them from the JavaScript code. Nodes highlighted red do not have direct references. They are only alive because they are part of the yellow node's tree. In general, you want to focus on the yellow nodes. Fix your code so that the yellow node isn't alive for longer than it needs to be, and you also get rid of the red nodes that are part of the yellow node's tree.

Click on a yellow node to investigate it further. In the Objects pane you can see more information about the code that's referencing it. For example, in the screenshot below you can see that the detachedTree variable is referencing the node. To fix this particular memory leak, you would study the code that uses detachedTree and ensure that it removes its reference to the node when it's no longer needed.

investigating a yellow node

Using Chrome DevTools To Hunt Javascript Memory Leaks

Using Chrome DevTools To Hunt Javascript Memory Leaks

In this section we will learn how to use the Chrome DevTools to identify javascript memory leaks in your code by making use of these 3 developer tools –

  1. Timeline View
  2. Heap Memory Profiler
  3. Allocation Timeline (or Allocation profiler)

First open any code editor of your choice and create an HTML doc with the code below and open it in chrome browser

<script>

       var foo = [];

       function grow() {

           foo.push(new Array(1000000).join('foo'));

           if (running)

               setTimeout(grow, 2000);

       }

       var running = false;


       $('#leak-button').click(function () {

           running = true;

           grow();

       });


       $('#stop-button').click(function () {

           running = false;

       });

   </script>


When the ‘Start’ button is clicked, it will call the grow() function which will append a string 1000000 characters long. The variable foo is a global variable which will not be garbage collected as it is being called by the grow() function recursively every second. Clicking the ‘Stop’ button will change the running flag to false to stop the recursive function call. Every time the function call ends, the garbage collector will free up memory but the variable foo will not be collected, leading to a memory leak scenario.

1. Timeline View

The first Chrome Developer Tool that we will put to use for identifying memory leaks is called ‘Timeline’. Timeline is a centralized overview of your code’s activity which helps you to analyze where time is spent on loading, scripting, rendering etc. You can visualize your memory leaks using the timeline recording option and compare memory usage data before and after the garbage collection.

  • Step1: Open our HTML doc in Chrome browser and press Ctrl+Shift+I to open Developer Tools.
  • Step2: Click on performance tab to open timeline overview window. Click Ctrl+E or click the record button to start timeline recording. Open your webpage and click on ‘start button’.
  • Step3: wait for 15 seconds and proceed to click ‘Stop button’ on your webpage. Wait for 10 seconds and click on garbage icon to the right to manually trigger garbage collector and stop the recording.

a

As you can see in the screenshot above, memory usage is going up with time. Every spike indicates when the grow function is called. But after the function execution ends, garbage collector clears up most of the garbage except the global foo variable. It keeps on increasing more memory and even after ending the program, the memory usage in the end did not drop to initial state.

2. Heap Memory Profiler

The ‘Heap Memory Profiler’ shows memory distribution by JavaScript objects and related DOM nodes. Use it to take heap snapshots, analyze memory graphs, compare snapshot data, and find memory leaks.

  • Step1 : Press Ctrl+Shift+I to open Chrome Dev Tools and click on memory panel.
  • Step2 : Select ‘Heap Snapshot’ option and click start.
  • a

  • Step3 : Click the start button on your webpage and select the record heap snapshot button at top left under memory panel. Wait for 10-15 seconds and click close button on your webpage. Proceed ahead and take a second heap snapshot.

    a

  • Step4 : select ‘comparison’ option from the drop down instead of ‘summary’ and search for detached DOM elements. This will help to identify Out of DOM references. There are none in our example case(the memory leak in our examle is due to global variable)

3.) Allocation Timeline/Profiler

The allocation profiler combines the snapshot information of the heap memory profiler with the incremental tracking of the Timeline panel. The tool takes heap snapshots periodically throughout the recording (as frequently as every 50 ms!) and one final snapshot at the end of the recording. Study the generated graph for suspicious memory allocation.

In newer versions of chrome, ‘Profiles’ tab has been removed. You can now find allocation profiler tool inside the memory panel rather than the profiles panel.

  • Step1 : Press Ctrl+Shift+I to open Chrome Dev Tools and click on memory panel.
  • Step2 : Select ‘Allocation Instrumentation on timeline’ option and click start.
  • a

  • Step 3: Click and record and wait for allocation profiler to automatically take snapshots in a periodical manner. Analyse the generated graph for suspicious memory allocation.
  • a

    a

Removing the memory leak by modifying our code

Now that we have successfully used chrome developer tools to identify the memory leak in our code, we need to tweak our code to eliminate this leak.

As discussed earlier in the ’causes of memory leaks’ section, we saw how global variables are never disposed of by garbage collectors especially when they are being recursively called by a function. We have 3 ways in which we can modify our code –

  1. Set the global variable foo to null after it is no longer needed.
  2. Use ‘let’ instead of ‘var’ for variable foo declaration. Let has a block scope unlike var. It will be garbage collected.
  3. Put the foo variable and the grow() function declarations inside the click event handler.
 <script>
       var running = false;

       $('#leak-button').click(function () {
           /* Variable foo and grow function are now decalred inside the click event handler. They no longer have global scope. They now have local scope and therefore will not lead to memory leak*/
           var foo = [];

           function grow() {
               foo.push(new Array(1000000).join('foo'));
               if (running)
                   setTimeout(grow, 2000);
           }
           running = true;
           grow();
       });

       $('#stop-button').click(function () {
           running = false;
       });
   </script>

Conclusion

It’s nearly impossible to completely avoid javascript memory leaks, especially in large applications. A minor leak will not affect an application’s performance in any significant manner. Moreover, modern browsers like Chrome and Firefox armed with advanced garbage collector algorithms do a pretty good job in eliminating memory leaks automatically. This doesn’t mean that a developer must be oblivious to efficient memory management. Good coding practices go a long way in curbing any chance of leaks right from the development phase to avoid complications later. Use Chrome Developer tools to identify as many javascript memory leaks as you can to deliver an amazing user experience free from any freezes or crashes.

Friday, June 5, 2020

Scala foldLeft and foldRight

According to my experience, one of the best ways to workout the intuition is to see how it works on the very simple examples:
List(1, 3, 8).foldLeft(100)(_ - _) : Explanation:((100 - 1) - 3) - 8 == 88
List(1, 3, 8).foldRight(100)(_ - _): Explanation:  1 - (3 - (8 - 100)) == -94
As you can see, foldLeft/Right just passes the element of the list and the result of the previous application to the the operation in second parentheses. It should be also mentioned that if you apply these methods to the same list, they will return equal results only if the applied operation is associative.
More examples
class Foo(val name: String, val age: Int, val sex: Symbol)

object Foo {
  def apply(name: String, age: Int, sex: Symbol) = new Foo(name, age, sex)
}

val fooList = Foo("Hugh Jass", 25, 'male) ::
              Foo("Biggus Dickus", 43, 'male) ::
              Foo("Incontinentia Buttocks", 37, 'female) ::
              Nil
              
              
val stringListFoldRight = fooList.foldRight(List[String]()) { ( f,z) =>
  val title = f.sex match {
    case 'male => "Mr."
    case 'female => "Ms."
  }
  z :+ s"$title ${f.name}, ${f.age}"
}

println(stringListFoldRight) 
//List(Ms. Incontinentia Buttocks, 37, Mr. Biggus Dickus, 43, Mr. Hugh Jass, 25)

val stringListFoldLeft = fooList.foldLeft(List[String]()) { (z,f) =>
  val title = f.sex match {
    case 'male => "Mr."
    case 'female => "Ms."
  }
  z :+ s"$title ${f.name}, ${f.age}"
}

println(stringListFoldLeft) 
//List(Mr. Hugh Jass, 25, Mr. Biggus Dickus, 43, Ms. Incontinentia Buttocks, 37)



Recent Post

Databricks Delta table merge Example

here's some sample code that demonstrates a merge operation on a Delta table using PySpark:   from pyspark.sql import SparkSession # cre...