Hello World! An Invitation to Computer Science

Lecture Slides: Algorithmic Efficiency

Monday, February 4, 2008 (and Thursday, February 7, 2008)

Key ideas from last time

• many algorithms operate on lists
• mailbox analogy and its limitations
• sequential search
• finding largest value requires non-empty list

Algorithms should be correct

• solving "right problem" v. solving "problem right"
• theory v. practice
    abstract mathematics → applied engineering
    algorithms should be useful in real world
• reusability; should be robust in face of:
  • repetition (should not work just by fluke)
  • differing inputs
  • change of scale
  • change of computing agent

Algorithms should be easy to understand

  input n
  sum ← 0
  i ← 1
  while i ≤ n:
    sum ← sum + i
    i ← i + 1
  output sum

use common idioms where possible

Algorithms should be elegant

compactness counts...

  input n
  sum ← (n * (n + 1)) / 2
  output sum

... but may be at odds with ease of understanding

Algorithms should be efficient!

• goal: minimize use of resources
    time, space(memory/storage), energy, money


• is march of progress enough to compensate for poor efficiency?
• why not measure efficiency with a stopwatch?
    too much variability due to computing agent
    (but benchmarking can be quite useful)


• focus on counting steps executed

Analysis of algorithms

• can be made formal: computational complexity
• worst case, best case, average case
• parametrization by size of input
• sequential search: size of list, not values on list
    not practical for really large lists

Order of magnitude

• growth rates, ignoring constant factors
• graphs: shape is most important
    we want to understand the trend
• how does it scale?
   if double input size, how much longer to wait?

Sorting & Searching

• maybe two most studied problems in c.s.
• used almost everywhere
• sorting puts data in order
• ordered data can be searched much faster

Functional abstraction

• blackboxing revisited: functions and procedures
• building blocks for other algorithms
• reuse of established algorithms
• idea of an algorithm "library"
• function calls can be single step in new algorithm
• need to factor in complexity of function

Selection sort

  input n; L1, ..., Ln
  u ← n
  while u > 0:
     p ← find position of max in L1, ..., Lu
     exchange Lp with Lu
     u ← u - 1

Exchange (a.k.a. Swap)

procedure: exchange places of two items on a list

  input n; L1, ..., Ln; x; y
  t ← Lx
  Lx ← Ly
  Ly ← t

requires 3 computational steps
  independent of size of input list
  constant-time algorithm: O(1)

how much space does procedure take?

Finding maxima revisited

function: find position in list where max occurs

  input n; L1, L2,  ... Ln
  pos ← 1
  i ← 2
  while i ≤ n:
    if Li > Lpos
      pos ← i
    i ← i + 1
  output pos

requires 3+4(n-1) computational steps in worst case
  linear-time algorithm: O(n)

Analyzing selection sort

• 2 peripheral steps
• n times:
    1 simple step
    1 exchange procedure (3 steps)
    1 find max position function (3+4(k-1) steps)
      where k is size of unsorted part of list
• simplifying and using Gauss's formula
    1 + 2 + ... + (n-1) = n(n-1)/2
• quadratic-time algorithm: O(n²)
• what if we just count comparisons?
• what if we just count exchanges?