7    VisuAlgo.net / /heap Login Binary Heap
Exploration Mode ▿

>

>
slow
fast
go to beginning previous frame pause play next frame go to end

A Binary (Max) Heap is a complete binary tree that maintains the Max Heap property.


Binary Heap is one possible data structure to model an efficient Priority Queue.


To focus the discussion scope, we design this visualization to show a Binary Max Heap that contains distinct integers only. However, as this visualization only accept integers, it is easy to convert a Binary Max Heap into a Binary Min Heap by re-creating a Binary Max Heap with the negation of every integer in the original Binary Max Heap.


Remarks: By default, we show e-Lecture Mode for first time (or non logged-in) visitor.
Please login if you are a repeated visitor or register for a free account first.

X Esc
Next
N.Sect ]

Complete Binary Tree: Every level in the binary tree, except possibly the last/lowest level, is completely filled, and all vertices in the last level are as far left as possible


Binary Max Heap property: The parent of each vertex - except the root - contains value greater than the value of that vertex. This is an easier-to-verify definition than the following alternative definition: The value of a vertex - except the leaf/leaves - must be greater than the value of its one (or two) child(ren).

X Esc
Prev
Next
N.Sect ]

Priority Queue (PQ) Abstract Data Type (ADT) is similar to normal Queue ADT, but with these two major operations:

  1. Enqueue(x): Put a new item (key) x into the PQ (in some order)
  2. y = Dequeue(): Return an existing item y that has the highest priority (key) in the PQ and if ties, return the one that is inserted first, i.e. back to First In First Out (FIFO) behavior of a normal queue

There are several potential usage of PQ ADT, e.g. Air Traffic Controller prioritizing certain flights to land first, Emergency situation in Hospital prioritizing urgent cases, etc. Discussion: Can you mention a few other real-life situations where a PQ is needed?


We are able to implement this PQ ADT using (circular) array but we will have either slow, i.e. O(N) Enqueue or Dequeue operation. Discussion: Why?

X Esc
Prev
Next
N.Sect ]

You can view the visualisation of a (random) Binary (Max) Heap here!


You should see a complete binary tree and all vertices except the root satisfy the Max Heap property (A[parent(i)] > A[i] — remember that we disallow duplicate integers).


Quiz: Based on this Binary (Max) Heap property, where will the largest integer be located?

At the root
At one of the leaf
Can be anywhere
X Esc
Prev
Next
[ P.Sect
N.Sect ]

Important fact to memorize at this point: If we have a Binary Heap of N items, since we will store it as a complete binary tree, its height will not be taller than O(log N)


Simple analysis: The size N of a full (more than just a complete) binary tree of height h is always N = 2^(h+1)-1, therefore h = log2(N+1)-1 ~= log2(N). This fact is important in the analysis of all Binary Heap-related operations.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

A complete binary tree can be stored efficiently as a compact array as there is no gap between vertices of a complete binary tree/elements of a compact array. To simplify the navigation operations below, we use 1-based array. VisuAlgo displays the index of each vertex as a red label below each vertex. Read those indices in sorted order from 1 to N, then you will see the vertices of the complete binary tree from top to down, left to right.


This way, we can implement basic binary tree traversal operations with simple index manipulations (with help of bit shift manipulation):

  1. parent(i) = i>>1, index i divided by 2 (integer division),
  2. left(i) = i<<1, index i multiplied by 2,
  3. right(i) = (i<<1)+1, index i multiplied by 2 and added by 1.
X Esc
Prev
Next
[ P.Sect
N.Sect ]

In this visualization, you can perform five (5) common/standard Binary (Max) Heap operations:

  1. Insert(v) in O(log N)
  2. ExtractMax() in O(log N)
  3. Create(A) - O(N log N) version
  4. Create(A) - O(N) version
  5. HeapSort() - in O(N log N)

There are others possible Binary (Max) Heap operations, but currently we do not elaborate them for pedagogical reason in a certain NUS module.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Insert(v): Insertion of a new item v into a Binary Max Heap can only be done at the last index N plus 1 to maintain the compact array = complete binary tree property. However, the Max Heap property may still be violated. This operation then fixes Max Heap property from the insertion point upwards (if necessary) and stop when there is no longer Max Heap property violation. Now try Insert(v) to the currently displayed Binary (Max Heap) (we randomize V every time).


Discussion 1: Do you know why swapping with parent when there is a Max Heap property violation during insertion is always a correct strategy?


Discussion 2: What is the time complexity of this Insert(v) operation?

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Answer 1: Think about Max Heap property before and after such swap, you should see that this swap preserves the correctness of the Max Heap property of affected vertices.


Answer 2/Analysis: The worst case of Insert(v) is when we insert a new item v that is greater than the value of the current root. Such insertion causes Insert(v) to fix Max Heap property from a leaf up to the root and therefore runs in O(log N) as a complete binary tree can never be taller than O(log N). Now try Insert(v) - Extreme where we will insert a new item that is always 1 more than the current root item.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

ExtractMax(): The reporting and then the deletion of the maximum item (the root) of a Binary Max Heap requires an existing item to replace the root, otherwise the Binary Max Heap becomes two disjoint subtrees. That item must be the last index N for the same reason: To maintain the compact array = complete binary tree property. Because we promote a leaf vertex to the root vertex of a Binary Max Heap, it will very likely violates the Max Heap property. This operation then fixes Binary Max Heap property from the root downwards by comparing the current value with the its child/the larger of its two children (if necessary). Now try ExtractMax() on the currently displayed Binary (Max) Heap.


Discussion 1: Do you know why if a vertex has two children, we have to check (and possibly swap) that vertex with the larger of its two children during the downwards fix of Max Heap property? Why can't we just compare with the left (or right, if exists) vertex only?


Discussion 2: What is the time complexity of this ExtractMax operation?

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Answer 1: You can construct a simple counter example that comparing with left vertex only (or right vertex only) can produce wrong answer, try [3,1,2] versus [3,2,1].


Answer 2/Analysis: This operation also runs in O(log N) as in the worst case, the last vertex (a leaf) happens to have the smallest value, try [7,6,5,4,3,2,1]. When it is promoted to the root, it will eventually trickle back to one of the leaf via a path that has up to O(log N) edges.


Now we have a data structure that can implement the two major operations of Priority Queue ADT: Enqueue(x) a.k.a. Insert(v) and y = Dequeue() a.k.a. ExtractMax() in efficient, O(log N) time. But we can do a few more operations.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Create(A): Creates a valid Binary (Max) Heap from an input array A of N integers (comma separated) into an initially empty Binary Max Heap.


There are two variants for this operations, one that is simpler but runs in O(N log N) and a more advanced technique that runs in O(N).


Pro tip: Try opening two copies of VisuAlgo on two browser windows. Execute different Create(A) versions on the worst case 'Sorted example' to see the somewhat dramatic differences of the two.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Create(A) - O(N log N): Simply insert (that is, by calling Insert(v) operation) all N integers of the input array into an initially empty Binary Max Heap one by one.


Analysis: This operation is clearly O(N log N) as we call O(log N) Insert(v) operation N times. Let's examine the 'Sorted example' which is the extreme case of this operation (Now try the Extreme Case - O(N log N) where we show a case where A=[1,2,3,4,5,6,7] -- please be patient as this example will take some time to complete). If we insert values in increasing order into an initially empty Binary Max Heap, then every insertion triggers a path from the insertion point (a new leaf) upwards to the root.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Create(A) - O(N): This faster version of Create(A) operation was invented by Robert W. Floyd in 1964. It takes advantage of the fact that a compact array = complete binary tree and all leaves (i.e. half of the vertices) are Binary Max Heap by default. This operation then fixes Binary Max Heap property (if necessary) only from the last internal vertex back to the root.


Analysis: A loose analysis gives another O(N/2 log N) = O(N log N) complexity but it is actually just O(2*N) = O(N) — details in the next few slides. Now try the Extreme Case - O(N) on the same input array A=[1,2,3,4,5,6,7] and see that on the same extreme case as with the previous slide, this operation is far superior than the O(N log N) version.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

Simple proof on why half of Binary (Max) Heap of N (without loss of generality, let's assume that N is even) items are leaves are as follows: Suppose that the last leaf is at index N, then the parent of that last leaf is at index i = N/2. The left child of vertex i+1, if exists, will be 2*(i+1) = 2*(N/2+1) = N+2, which exceeds index N (the last leaf) so index i+1 must also be a leaf vertex that has no child. That is, basically indices [i+1 = N/2+1, i+2 = N/2+2, ..., N], or half of the vertices, are leaves.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

The details are currently hidden as this requires heavy mathematical analysis.

X Esc
Prev
Next
[ P.Sect
N.Sect ]

HeapSort(): John William Joseph Williams invented HeapSort() algorithm in 1964, together with this Binary Heap data structure. HeapSort() operation (assuming the Binary Max Heap has been created in O(N)) is very easy. Simply call the O(log N) ExtractMax() operation N times. Now try HeapSort() on the currently displayed Binary (Max) Heap.


Simple Analysis: HeapSort() clearly runs in O(N log N) — an optimal comparison-based sorting algorithm.


Quiz: In best case scenario, HeapSort() is asymptotically faster than...

Merge Sort
Bubble Sort
Insertion Sort
Selection Sort
X Esc
Prev
Next
[ P.Sect
N.Sect ]

Although HeapSort() runs in θ(N log N) time for all (best/average/worst) cases, is it really the best comparison-based sorting algorithm?


Discussion point: How about caching performance of HeapSort()?

X Esc
Prev
Next
[ P.Sect
N.Sect ]

You have reached the end of the basic stuffs of this Binary (Max) Heap data structure. However, we still have a few more interesting Binary (Max) Heap challenges for you that are outlined in this section.


When you have cleared them all, we invite you to study more advanced algorithms that use Priority Queue as (one of) its underlying data structure, like Prim's MST algorithm, Dijkstra's SSSP algorithm, etc.

X Esc
Prev
Next
[ P.Sect

If you are looking for an implementation of Binary (Max) Heap to actually model a Priority Queue, then there is a good news.


C++ and Java already have built-in Priority Queue implementations that very likely use this data structure. They are C++ STL priority_queue (the default is a Max Priority Queue) and Java PriorityQueue (the default is a Min Priority Queue).


However, the built-in implementation may not be suitable to do some Priority Queue extended operations efficiently (details omitted) for pedagogical reason in a certain NUS module.

X Esc
Prev
Next
[ P.Sect

For a few more interesting questions about this data structure, please practice on Binary Heap training module (no login is required, but short and of medium difficulty setting only).


However, for registered users, you should login and then go to the Main Training Page to officially clear this module and such achievement will be recorded in your user account.

X Esc
Prev
Next
[ P.Sect

We also have a few programming problems that somewhat requires the usage of this Binary Heap data structure: UVa 01203 - Argus and Kattis - numbertree.


Try them to consolidate and improve your understanding about this data structure. You are allowed to use C++ STL priority_queue or Java PriorityQueue if that simplifies your implementation.

X Esc
Prev
Next
[ P.Sect

This slide is currently hidden and will only be used in a live lecture :).

X Esc
Prev
Next
[ P.Sect

As the action is being carried out, each step will be described in the status panel.

X Esc
Prev
Next

You can also follow the pseudocode highlights to trace the algorithm.

X Esc
Prev
Next

Control the animation with the player controls! Keyboard shortcuts are:

Spacebar: play/pause/replay
Left/right arrows: step backward/step forward
-/+: decrease/increase speed
X Esc
Prev
Next

Return to 'Exploration Mode' to start exploring!


Note that if you notice any bug in this visualization or if you want to request for a new visualization feature, do not hesitate to drop an email to the project leader: Dr Steven Halim via his email address: stevenhalim at gmail dot com.

X Esc
Prev

Create(A) - O(N log N)

Create(A) - O(N)

Insert(v)

ExtractMax()

HeapSort()

>

Go

Sorted Example

Random

Go

Sorted Example

Random

Go

About Team Terms of use

About

VisuAlgo was conceptualised in 2011 by Dr Steven Halim as a tool to help his students better understand data structures and algorithms, by allowing them to learn the basics on their own and at their own pace.

VisuAlgo contains many advanced algorithms that are discussed in Dr Steven Halim's book ('Competitive Programming', co-authored with his brother Dr Felix Halim) and beyond. Today, some of these advanced algorithms visualization/animation can only be found in VisuAlgo.

Though specifically designed for National University of Singapore (NUS) students taking various data structure and algorithm classes (e.g. CS1010, CS1020, CS2010, CS2020, CS3230, and CS3230), as advocators of online learning, we hope that curious minds around the world will find these visualisations useful too.

VisuAlgo is not designed to work well on small touch screens (e.g. smartphones) from the outset due to the need to cater for many complex algorithm visualizations that require lots of pixels and click-and-drag gestures for interaction. The minimum screen resolution for a respectable user experience is 1024x768 and only the landing page is relatively mobile-friendly.

VisuAlgo is an ongoing project and more complex visualisations are still being developed.

The most exciting development is the automated question generator and verifier (the online quiz system) that allows students to test their knowledge of basic data structures and algorithms. The questions are randomly generated via some rules and students' answers are instantly and automatically graded upon submission to our grading server. This online quiz system, when it is adopted by more CS instructors worldwide, should technically eliminate manual basic data structure and algorithm questions from typical Computer Science examinations in many Universities. By setting a small (but non-zero) weightage on passing the online quiz, a CS instructor can (significantly) increase his/her students mastery on these basic questions as the students have virtually infinite number of training questions that can be verified instantly before they take the online quiz. The training mode currently contains questions for 12 visualization modules. We will soon add the remaining 8 visualization modules so that every visualization module in VisuAlgo have online quiz component.

Another active branch of development is the internationalization sub-project of VisuAlgo. We want to prepare a database of CS terminologies for all English text that ever appear in VisuAlgo system. This is a big task and requires crowdsourcing. Once the system is ready, we will invite VisuAlgo visitors to contribute, especially if you are not a native English speaker. Currently, we have also written public notes about VisuAlgo in various languages: zh, id, kr, vn, th.

Team

Project Leader & Advisor (Jul 2011-present)
Dr Steven Halim, Senior Lecturer, School of Computing (SoC), National University of Singapore (NUS)
Dr Felix Halim, Software Engineer, Google (Mountain View)

Undergraduate Student Researchers 1 (Jul 2011-Apr 2012)
Koh Zi Chun, Victor Loh Bo Huai

Final Year Project/UROP students 1 (Jul 2012-Dec 2013)
Phan Thi Quynh Trang, Peter Phandi, Albert Millardo Tjindradinata, Nguyen Hoang Duy

Final Year Project/UROP students 2 (Jun 2013-Apr 2014)
Rose Marie Tan Zhao Yun, Ivan Reinaldo

Undergraduate Student Researchers 2 (May 2014-Jul 2014)
Jonathan Irvin Gunawan, Nathan Azaria, Ian Leow Tze Wei, Nguyen Viet Dung, Nguyen Khac Tung, Steven Kester Yuwono, Cao Shengze, Mohan Jishnu

Final Year Project/UROP students 3 (Jun 2014-Apr 2015)
Erin Teo Yi Ling, Wang Zi

Final Year Project/UROP students 4 (Jun 2016-Dec 2017)
Truong Ngoc Khanh, John Kevin Tjahjadi, Gabriella Michelle, Muhammad Rais Fathin Mudzakir

List of translators who have contributed ≥100 translations can be found at statistics page.

Acknowledgements
This project is made possible by the generous Teaching Enhancement Grant from NUS Centre for Development of Teaching and Learning (CDTL).

Terms of use

VisuAlgo is free of charge for Computer Science community on earth. If you like VisuAlgo, the only payment that we ask of you is for you to tell the existence of VisuAlgo to other Computer Science students/instructors that you know =) via Facebook, Twitter, course webpage, blog review, email, etc.

If you are a data structure and algorithm student/instructor, you are allowed to use this website directly for your classes. If you take screen shots (videos) from this website, you can use the screen shots (videos) elsewhere as long as you cite the URL of this website (http://visualgo.net) and/or list of publications below as reference. However, you are NOT allowed to download VisuAlgo (client-side) files and host it on your own website as it is plagiarism. As of now, we do NOT allow other people to fork this project and create variants of VisuAlgo. Using the offline copy of (client-side) VisuAlgo for your personal usage is fine.

Note that VisuAlgo's online quiz component is by nature has heavy server-side component and there is no easy way to save the server-side scripts and databases locally. Currently, the general public can only use the 'training mode' to access these online quiz system. Currently the 'test mode' is a more controlled environment for using these randomly generated questions and automatic verification for a real examination in NUS. Other interested CS instructor should contact Steven if you want to try such 'test mode'.

List of Publications

This work has been presented briefly at the CLI Workshop at the ACM ICPC World Finals 2012 (Poland, Warsaw) and at the IOI Conference at IOI 2012 (Sirmione-Montichiari, Italy). You can click this link to read our 2012 paper about this system (it was not yet called VisuAlgo back in 2012).

This work is done mostly by my past students. The most recent final reports are here: Erin, Wang Zi, Rose, Ivan.

Bug Reports or Request for New Features

VisuAlgo is not a finished project. Dr Steven Halim is still actively improving VisuAlgo. If you are using VisuAlgo and spot a bug in any of our visualization page/online quiz tool or if you want to request for new features, please contact Dr Steven Halim. His contact is the concatenation of his name and add gmail dot com.