From f4828d79afa26bbdd393893a83b1ae61e87d6344 Mon Sep 17 00:00:00 2001 From: Dean Wampler Date: Wed, 1 Dec 2010 21:47:08 -0600 Subject: [PATCH] Final draft of course notes! --- .../html/all.html | 334 ++++++++++-------- .../html/lecture1.html | 103 +++--- .../html/lecture13.html | 2 +- .../html/lecture14.html | 15 +- .../html/references.html | 3 +- .../slides/lecture1.slides | 7 +- .../slides/lecture14.slides | 15 +- 7 files changed, 271 insertions(+), 208 deletions(-) diff --git a/PragmaticsOfIndustrialSWDevelopment/html/all.html b/PragmaticsOfIndustrialSWDevelopment/html/all.html index 838bf01..e130106 100644 --- a/PragmaticsOfIndustrialSWDevelopment/html/all.html +++ b/PragmaticsOfIndustrialSWDevelopment/html/all.html @@ -101,10 +101,10 @@

Contents

  • Lecture 12: Complexity vs. Simplicity Part I: Effective Software Development Processes – Scrum, XP, and Lean
  • -h1. Course Components: Project Exercises (cont.) + +
    +

    Course Components: Project Exercises (cont.)


    -
    +
  • Topics to be worked out with me.
  • +
    +

    Midterm and Final Exam

    @@ -315,9 +332,9 @@

    Miscellaneous

    Scaling Computation, Part I

      -
    • Two models of concurrency compared:
      - * Multithreaded Programming.
      - * The Actor Model of concurrency.
    • +
    • Two models of concurrency compared:
    • +
    • Multithreaded Programming.
    • +
    • The Actor Model of concurrency.
    @@ -349,7 +366,7 @@

    Is This Safe?

    public int getNext() { return value++; } -}

    +}

    code/UnsafeSequence.java

    (credit: all Multithreading examples adapted from “Java Concurrency in Practice”)

    @@ -363,7 +380,7 @@

    This Is Safe

    public synchronized int getNext() { return value++; } -}

    code/SafeSequence.java

    +}

    code/SafeSequence.java

    Things Can Go Wrong

    @@ -395,7 +412,7 @@

    Deadlock Example

    synchronized (right) { synchronized (left) { doSomethingElse(); } } -}

    code/LeftRightDeadlock.java

    +}

    code/LeftRightDeadlock.java

    Other Deadlock Hazards

    @@ -417,11 +434,10 @@

    Livelock

    A thread is not blocked, but it can’t complete work because of trying operations always fails.


    -
    +
  • One scenario is two contenders with identical strategies for trying again, often colliding in the same way. If one is greedy, both may succeed.
  • +
    +

    The Importance of Immutability

    What about access to immutable state? All the synchronization issues go away. Immutable objects have these virtues:

    code/manager-actor.scala

    +

    code/manager-actor.scala

    Actors in Scala (Akka)

    @@ -506,7 +522,7 @@

    Starting an Akka Actor

    import se.scalablesolutions.akka.actor.Actor_ // Creating an actor (one way...) and starting val manager = Actor.actorOf[Manager] -manager.start

    code/manager-actor2.scala

    +manager.start

    code/manager-actor2.scala

    Actor vs. ActorRef

    @@ -526,7 +542,7 @@

    Sending a Message to an Akka Actor

    case class Unregister(actor: ActorRef) case object Stop case object Start -

    code/actor-messages.scala

    +

    code/actor-messages.scala

    An Asynchronous (Fire and Forget) Message

    @@ -539,7 +555,7 @@

    An Asynchronous (Fire and Forget) Message

    case Start => managedActors foreach (_ ! Start) } } -

    code/actor-asynch-message.scala

    +

    code/actor-asynch-message.scala

    If a response is required, the receiver can send a message to the sender.

    @@ -556,7 +572,7 @@

    An Synchronous (Wait) Message

    } } } -

    code/actor-synch-message.scala

    +

    code/actor-synch-message.scala

    An Synchronous (Wait) Message (cont.)

    @@ -579,7 +595,7 @@

    An Asynchronous with a Future Message

    case None => "Error!" } } -

    code/actor-future-message.scala

    +

    code/actor-future-message.scala

    An Asynchronous with a Future Message (cont.)

    @@ -603,7 +619,7 @@

    Another Way to Write the Last Example

    case None => "Error!" } } -

    code/actor-future-message2.scala

    +

    code/actor-future-message2.scala

    Achieving Immutability

    @@ -763,9 +779,8 @@

    Add New Functionality (cont.)

    Add New Functionality (cont – sort of…)

    How could we modify filter to filter by instruments? (To make it easier, assume there is only one instrument in the list, as in the test.)

    This is not easy, as we really need to parse the JSON into some representation, filter out the records (which are of the form {"timestamp": 1283233404995, "symbol": "A", "price", "40.0"}) using the symbol, then convert back to a string.

    -

    Can you figure out how to do this with the Lift JSON APIs used in the application and JSON helper types provided, or using other 3rd-party JSON libraries?
    -
    -Actually, the right way to implement this feature is to support this filtering at the database level, which we will learn how to do with MongoDB.

    +

    Can you figure out how to do this with the Lift JSON APIs used in the application and JSON helper types provided, or using other 3rd-party JSON libraries?

    +

    Actually, the right way to implement this feature is to support this filtering at the database level, which we will learn how to do with MongoDB.

    Lecture 2: Scaling Computation, Part II, and Course Project Rebooted

    @@ -1004,7 +1019,7 @@

    Symantics (Using Akka Syntax)

    counter // -> 1 counter // -> 2 -ref.get // -> 2 (retrieve the value explicitly)

    code/akka-stm-example1.scala

    +ref.get // -> 2 (retrieve the value explicitly)

    code/akka-stm-example1.scala

    Symantics (cont.)

    @@ -1019,7 +1034,7 @@

    Symantics (cont.)

    atomic { ref set 10 // set new value } // (actually returns old value) -ref.get // -> 10

    code/akka-stm-example2.scala

    +ref.get // -> 10

    code/akka-stm-example2.scala

    Symantics: Blocking (“Retry”)

    @@ -1040,7 +1055,7 @@

    Symantics: Blocking (“Retry”)

    to alter (_ + amount) } } -

    code/akka-stm-example3.scala

    +

    code/akka-stm-example3.scala

    (See Blocking Transactions on the Akka STM page for the full example.)

    @@ -1065,7 +1080,7 @@

    Symantics: Choice (“orElse”)

    } } } -

    code/akka-stm-example4.scala

    +

    code/akka-stm-example4.scala

    Composability and Modularity

    @@ -1188,7 +1203,7 @@

    Add New Functionality (cont.)

    [error] Test Failed: calculateStatistics returns a JSON string containing all data that matches the instrument criteria
     org.scalatest.TestFailedException: JObject(List(JField(criteria,JObject(List(JField(instruments,JArray(List(JString(A)))), JField(statistics,JArray(List(JString(price[$])))), JField(start,JInt(0)), JField(end,JInt(1283824964864))))), JField(results,JArray(List(JObject(List(JField(timestamp,JInt(1283824954864)), JField(symbol,JString(A)), JField(price,JDouble(0.0)))), JObject(List(JField(timestamp,JInt(1283824955864)), JField(symbol,JString(C)), JField(price,JDouble(10.0)))), JObject(List(JField(timestamp,JInt(1283824956864)), JField(symbol,JString(A)), JField(price,JDouble(20.0)))), JObject(List(JField(timestamp,JInt(1283824957864)), JField(symbol,JString(B)), JField(price,JDouble(30.0)))), JObject(List(JField(timestamp,JInt(1283824958864)), JField(symbol,JString(A)), JField(price,JDouble(40.0))))))))) did not equal JObject(List(JField(criteria,JObject(List(JField(instruments,JArray(List(JString(A)))), JField(statistics,JArray(List(JString(price[$])))), JField(start,JInt(0)), JField(end,JInt(1283824964864))))), JField(results,JArray(List(JObject(List(JField(timestamp,JInt(1283824954864)), JField(symbol,JString(A)), JField(price,JDouble(0.0)))), JObject(List(JField(timestamp,JInt(1283824958864)), JField(symbol,JString(A)), JField(price,JDouble(40.0)))), JObject(List(JField(timestamp,JInt(1283824956864)), JField(symbol,JString(A)), JField(price,JDouble(20.0)))))))))
    -

    +

    code/exercise1-test-failure.scala

    (Yeah!)

    @@ -1218,11 +1233,10 @@

    Add New Functionality (cont.)

    ~test-only org.chicagoscala.awse.server.finance.InstrumentAnalysisServerTest


    -
    +
  • Using the ~test-only “action” saves time, running in a few seconds, while running all the tests takes longer.
  • +
    +

    Add New Functionality (cont.)

    +
    +

    Add New Functionality (step 6)

    Key-Value Stores

    -

    E.g., Amazon SimpleDB, Riak, Redis, Tokyo Cabinet, Scalaris, MemcacheDB, BerkeleyDB, MNesia.
    -
    -Think of hash maps on steroids.

    +

    E.g., Amazon SimpleDB, Riak, Redis, Tokyo Cabinet, Scalaris, MemcacheDB, BerkeleyDB, MNesia.

    +

    Think of hash maps on steroids.

    “Polyglot Persistence”

    -

    A big trend today is that non-trivial systems mix different programming languages, libraries and tools. Do we use just one language to write all parts of a standard web application? No.1

    +

    A big trend today is that non-trivial systems mix different programming languages, libraries and tools. Do we use just one language to write all parts of a standard web application? No.1

    The same phenomenon is now happening at the persistence level. Often, one persistence strategy doesn’t fit all needs in complex, distributed systems. You might use a fast, key-value store for user sessions, then persist key “events” to an RDBMS, for example.

    -

    1 Well, maybe we could with JavaScript…

    +

    1 Well, maybe we could with JavaScript…

    Some Techniques We’ll Discuss

    @@ -2603,9 +2614,8 @@

    Conclusions: MongoDB vs. CouchDB

    Key-Value Stores

    -

    E.g., Amazon SimpleDB, Riak, Redis, Tokyo Cabinet, Scalaris, MemcacheDB, BerkeleyDB, Mnesia.
    -
    -Think of hash maps on steroids.

    +

    E.g., Amazon SimpleDB, Riak, Redis, Tokyo Cabinet, Scalaris, MemcacheDB, BerkeleyDB, Mnesia.

    +

    Think of hash maps on steroids.

    +

    When a row’s key is hashed, the node with the closest key greater than the row’s key is assigned the responsibility for coordinating reads and writes.

    Reads and Writes

    @@ -4015,15 +4024,15 @@

    Good Object Design: Lesson 2

    -

    1 C++ and C# have destructors for this purpose (but they may be invoked too late for some resources, like database connections).
    -2 Java has finalizers, but they are effectively useless, maybe even harmful.

    +

    1 C++ and C# have destructors for this purpose (but they may be invoked too late for some resources, like database connections).
    +2 Java has finalizers, but they are effectively useless, maybe even harmful.

    -

    Design by Contract1

    +

    Design by Contract1

    Design by Contract (DbC) is a programming discipline where you specify assertions about program behavior that the language infrastructure can enforce during testing, but disable in production deployments, for performance reasons.

    -

    1 A registered trademark of Eiffel Software

    +

    1 A registered trademark of Eiffel Software

    Design by Contract (cont.)

    @@ -4038,9 +4047,8 @@

    Design by Contract (cont.)

    Preconditions

    Preconditions are what must be true in order for the function to do its job. A trivial example: there must be enough free memory for the function to allocate the objects it creates.

    -

    Hence, these are requirements on the runtime environment and the clients using the function. Here is an example using our Windowing API and a “mythical” DbC framework that uses annotations.
    -
    -

    +

    Hence, these are requirements on the runtime environment and the clients using the function. Here is an example using our Windowing API and a “mythical” DbC framework that uses annotations.

    +
     
     package gui.windowing {
    @@ -4056,9 +4064,8 @@ 

    Preconditions

    Postconditions

    Postconditions are what the function guarantees to satisfy when it finishes its job, if the preconditions are true. A trivial example: a function might promise to never return null.

    -

    Hence, these are requirements on the function itself, constituting promises to clients. Here is an example using our Windowing API.
    -
    -

    +

    Hence, these are requirements on the function itself, constituting promises to clients. Here is an example using our Windowing API.

    +
     
     package gui.windowing {
    @@ -4074,9 +4081,8 @@ 

    Postconditions

    Invariants

    -

    Invariants must be true before and after function invocation. They may specify some object (or global) property that doesn’t change. They may specify a logical constraint that must remain true, etc. Invariants can be requirements on the function, the environment, or both.
    -
    -

    +

    Invariants must be true before and after function invocation. They may specify some object (or global) property that doesn’t change. They may specify a logical constraint that must remain true, etc. Invariants can be requirements on the function, the environment, or both.

    +
     
     package gui.windowing {
    @@ -4115,6 +4121,7 @@ 

    Classes

    Scala Has Classes (for Example)

    Scala example, patterned after Java’s Exception Hierarchy:

    +
     
     class Exception(
    @@ -4285,6 +4292,7 @@ 

    Prototype-Based Languages

    Prototype-based languages don’t have classes. (Hence they are also called classless languages.) Instead, you just build up objects “from scratch”.

    JavaScript object-literal notation (essentially, a hash map):

    +
     
     var exception = {
    @@ -4309,8 +4317,8 @@ 

    Prototype-Based Languages (cont.)

    Aside: Open Classes

    Note that many dynamically-typed, class-based languages, e.g., Ruby, allow this kind of object modification, too. They are said to have open classes.

    -

    Most statically-typed languages, like Scala, Java, C#, and C++, have closed classes. You can’t add or remove members in objects.1

    -

    1 There are some workarounds, both byte-code level hacks and some language features, like Scala’s implicits.

    +

    Most statically-typed languages, like Scala, Java, C#, and C++, have closed classes. You can’t add or remove members in objects.1

    +

    1 There are some workarounds, both byte-code level hacks and some language features, like Scala’s implicits.

    Prototype-Based Languages (cont.)

    @@ -4496,6 +4504,7 @@

    Unapply and Pattern Matching

    Option[+A]: case class (cont.)

    So, adding case to the declaration is equivalent to this:

    +
     
     final class Some[+A](val x: A) extends Option[+A] {
    @@ -4530,6 +4539,7 @@ 

    Option[+A]: Nothing

    Nothing: A subtype of all other types, with no instances.

    Used for proper type safety.

    +
     
     case object None extends Option[Nothing] {...}
    @@ -4763,9 +4773,8 @@ 

    Uniform-Access Principle

    Uniform-Access Principle (cont.)

    -

    The caller doesn’t care whether a method is invoked or a bare field is accessed.
    -
    -The implementer might want to start with a simple bare field access, then eventually replace it with a method call (e.g., to do lazy initialization, etc.).

    +

    The caller doesn’t care whether a method is invoked or a bare field is accessed.

    +

    The implementer might want to start with a simple bare field access, then eventually replace it with a method call (e.g., to do lazy initialization, etc.).

    The uniform-access principle means the caller’s code doesn’t have to change, because the syntax is identical. (The code might need a recompile).

    @@ -4872,6 +4881,7 @@

    gui.windowing

    Other Modules

    BAD:

    +
     
     object MyApp {
    @@ -4883,6 +4893,7 @@ 

    Other Modules

    GOOD:

    +
     
     object MyApp {
    @@ -5544,8 +5555,8 @@ 

    Lecture 10: Functional Programming (FP): Into the Real World

    What Is Functional Programming?

    FP treats computation as the application of functions rather than the direct mutation of state.

    -

    FP has its roots in mathematics, specifically lambda calculus1, a formal system developed in the 1930s to investigate function definition, function application, recursion, etc. (Yes, before the invention of electronic computers.) Many functional programming languages can be viewed as elaborations on the lambda calculus.

    -

    1 For this reason, anonymous functions in many languages are called lambdas.

    +

    FP has its roots in mathematics, specifically lambda calculus1, a formal system developed in the 1930s to investigate function definition, function application, recursion, etc. (Yes, before the invention of electronic computers.) Many functional programming languages can be viewed as elaborations on the lambda calculus.

    +

    1 For this reason, anonymous functions in many languages are called lambdas.

    (Adapted from Wikipedia.)

    @@ -5577,9 +5588,9 @@

    Immutability Benefits

    No Side Effects

    Just as x is not mutable, so to is the rest of the world. So, by definition, immutability means there can be no side effects. This also has important implications for functions:

    x = cos(y)2

    -

    All work done by cos(y) (for example) is returned and assigned to x. there is no global state that is updated.1

    +

    All work done by cos(y) (for example) is returned and assigned to x. there is no global state that is updated.1

    Functions without side effects are called pure.

    -

    1 However, a real implementation might update “invisible” state like a cache of previously-calculated values.

    +

    1 However, a real implementation might update “invisible” state like a cache of previously-calculated values.

    Benefits of No Side Effects

    @@ -5699,8 +5710,8 @@

    Higher-Kinded Types (cont.)

    // => two // => 3.3
    -

    Note that we use the wildcard _ for the List[_] parameter, because we don’t care what it is.1

    -

    1 Actually, on the JVM we have no choice, because this type information is erased in the byte code.

    +

    Note that we use the wildcard _ for the List[_] parameter, because we don’t care what it is.1

    +

    1 Actually, on the JVM we have no choice, because this type information is erased in the byte code.

    Higher-Kinded Types (cont.)

    @@ -5902,8 +5913,8 @@

    OCP and The Expression Problem (cont.)

    OCP and The Expression Problem (cont.)

    The object-oriented approach is to use inheritance, where clients depend only on a well-defined abstraction, which is implemented by concrete subtypes. New “behaviors” can be added by defining new concrete subtypes for the same abstraction.

    -

    As long as the abstraction is sufficiently expressive and clients don’t depend on the concrete subtypes, we can add new behaviors to the code base without modifying existing code.1

    -

    1 Recall that, every now and then, the abstraction won’t be general enough to support a new behavior, so the abstraction and possibly all the subtypes will require modification.

    +

    As long as the abstraction is sufficiently expressive and clients don’t depend on the concrete subtypes, we can add new behaviors to the code base without modifying existing code.1

    +

    1 Recall that, every now and then, the abstraction won’t be general enough to support a new behavior, so the abstraction and possibly all the subtypes will require modification.

    OCP and The Expression Problem (cont.)

    @@ -6384,7 +6395,7 @@

    Summarized Together

    - + @@ -6431,7 +6442,7 @@

    Trends (cont.)

    Pre-Internet Internet Big Data
    - + @@ -6452,7 +6463,7 @@

    Trends (cont.)

    Pre-Internet Internet Big Data
    - + @@ -6479,7 +6490,7 @@

    Trends (cont.)

    Pre-Internet Internet Big Data
    - + @@ -6512,7 +6523,7 @@

    Trends (cont.)

    Pre-Internet Internet Big Data
    - + @@ -6551,7 +6562,7 @@

    Trends (cont.)

    Pre-Internet Internet Big Data
    - + @@ -6619,13 +6630,13 @@

    OO Design vs. Functional Design

    Software Reuse

    I’ll show an example later that compares modeling domain concepts with classes vs. representing objects as maps (which is how objects are represented in languages like JavaScript and Perl).

    -

    There are some significant benefits for reuse when using such functional data structures1.

    +

    There are some significant benefits for reuse when using such functional data structures1.

    • These types are very well tested.
    • They provide a variety of functions for computations over the collections.
    • They define a few usage protocols that are flexible and powerful.
    -

    1 As opposed to the dumbed-down versions found in Java, etc.

    +

    1 As opposed to the dumbed-down versions found in Java, etc.

    Software Reuse (cont.)

    @@ -6812,7 +6823,7 @@

    AOP: An Example Problem

    case class CheckingAccount(customer: Customer, var balance: Money) extends Account case class SavingsAccount(customer: Customer, var balance: Money, var interestRate: Double) extends Account -

    +

    AOP: An Example Problem (cont.)

    @@ -6835,7 +6846,7 @@

    AOP: An Example Problem (cont.)

    // Checking: initial account: CheckingAccount(Customer(Dean),Money(1000.0)) // Checking: final account: CheckingAccount(Customer(Dean),Money(1500.0)) // Savings: initial account: SavingsAccount(Customer(Dean),Money(2000.0),0.02) -// Checking: final account: SavingsAccount(Customer(Dean),Money(900.0),0.3)

    +// Checking: final account: SavingsAccount(Customer(Dean),Money(900.0),0.3)

    AOP and Modularity

    @@ -6857,7 +6868,7 @@

    AOP and Modularity

    _name = newName putToDatabase("name", _name) } -}

    +}

    Modularity Breakdown

    @@ -6883,10 +6894,10 @@

    Cross-Cutting Concerns

    Enterprise Java Beans

    -

    Enterprise Java Beans (EJBs) where a flawed attempt to address this problem (the flaws were mostly fixed in the v3.0 redo1). This model let you specify some data for persistence, transactions, security, etc. in XML configuration files, but you still had very invasive code in your domain objects.

    +

    Enterprise Java Beans (EJBs) where a flawed attempt to address this problem (the flaws were mostly fixed in the v3.0 redo1). This model let you specify some data for persistence, transactions, security, etc. in XML configuration files, but you still had very invasive code in your domain objects.

    Specifically, while the model “separated concerns” such as persistence from the domain model, the model objects needed to subclass EJB container types, implement methods to support lifecycle management, etc.

    That is, EJBs traded one set of problems for another set.

    -

    1 But only after the Spring Framework and the AOP community showed a better way.

    +

    1 But only after the Spring Framework and the AOP community showed a better way.

    Aspect-Oriented Programming

    @@ -6914,9 +6925,9 @@

    Characteristics of AOP Solutions

    An aspect framework has two parts.

    • Quantification: The ability to specify, in a succinct way (programmatically or declaratively), which “points of execution” to observe or modify. It’s analogous to a query language over the set of execution points.
    • -
    • Behavior Modification: Once the set of execution points is identified, you need ways to observe or modify the behavior, without modifying the original code1.
    • +
    • Behavior Modification: Once the set of execution points is identified, you need ways to observe or modify the behavior, without modifying the original code1.
    -

    1 I.e., this a special case of the Expression Problem, a.k.a Open-Closed Principle.

    +

    1 I.e., this a special case of the Expression Problem, a.k.a Open-Closed Principle.

    AOP Terminology

    @@ -6950,7 +6961,7 @@

    Code for AOP

    customer.name = newName putToDatabase("name", newName) } -}

    +}

    AOP Tools

    @@ -6986,13 +6997,13 @@

    AspectJ Example

    case None => } } -}

    +}

    Aspects Beyond Cross-Cutting Concerns

    -

    Note that the cross-cutting concerns that we mentioned, persistence, transactions, security, and logging, are sometimes called non-functional requirements, because they mostly relate to implementation concerns, rather than business requirements.1

    +

    Note that the cross-cutting concerns that we mentioned, persistence, transactions, security, and logging, are sometimes called non-functional requirements, because they mostly relate to implementation concerns, rather than business requirements.1

    Is this the only use for aspects?

    -

    1 This is not really an accurate distinction; everything we do is ultimately tied in some way to business requirements.

    +

    1 This is not really an accurate distinction; everything we do is ultimately tied in some way to business requirements.

    Aspects Beyond Cross-Cutting Concerns

    @@ -7054,15 +7065,15 @@

    “Advice” in Scala, Using Traits

    case class Customer(var name: String) extends Named -val dean = new Customer("Dean") with PersistentName

    +val dean = new Customer("Dean") with PersistentName

    Why Isn’t AOP Pervasive? (cont.)

    So we can get advice without aspects, although it’s still tedious to write all these special cases. However, we still don’t have quantification.

    Functional data structures can address both limitations.

    Objects are just ad hoc groupings of collections and “atoms” (numbers, strings, etc.), recursively. More specifically, they work like maps and in languages like JavaScript, they are maps!

    -

    So, if we create a PersistentMap1, in this case, and represent our objects as maps, we’ve solved the quantification problem!

    -

    1 In this case, meaning persistent to a database, not as in the STM case.

    +

    So, if we create a PersistentMap1, in this case, and represent our objects as maps, we’ve solved the quantification problem!

    +

    1 In this case, meaning persistent to a database, not as in the STM case.

    Why Isn’t AOP Pervasive? (cont.)

    @@ -7089,13 +7100,13 @@

    Why Isn’t AOP Pervasive? (cont.)

    } val dean = new HashMap[String,Any] with PersistentMap[String,Any] -dean.put("name", "Dean")

    +dean.put("name", "Dean")

    Why Isn’t AOP Pervasive? (cont.)

    What this means is that if we just use Maps (and perhaps a few other fundamental collections), we can instrument them as needed with less effort than instrumenting lots of ad-hoc classes. The quantification problem is reduced.

    -

    The key point here is that working with more “primitive” building blocks; functions, and core data structures, it is actually easier to generate the variations of behavior we need.1 We’ll return to this point in a few weeks when we talk about simplicity in systems, etc.

    -

    1 We can make the behaviors more composable using advanced techniques like monads.

    +

    The key point here is that working with more “primitive” building blocks; functions, and core data structures, it is actually easier to generate the variations of behavior we need.1 We’ll return to this point in a few weeks when we talk about simplicity in systems, etc.

    +

    1 We can make the behaviors more composable using advanced techniques like monads.

    Conclusions: OOP vs. FP (vs. AOP)

    @@ -7331,9 +7342,9 @@

    Waterfall Model of Software Development

    Waterfall Model of Software Development

    -

    The Waterfall Process resulted from a historical accident.1 During the 60s, many researches recognized the importance of iterative development. One paper was “Managing the Development of Large Software Systems”, by Winston Royce, in Proceedings of IEEE Westcon.

    +

    The Waterfall Process resulted from a historical accident.1 During the 60s, many researches recognized the importance of iterative development. One paper was “Managing the Development of Large Software Systems”, by Winston Royce, in Proceedings of IEEE Westcon.

    This paper started with an image similar to the one two slides ago, but it went on to argue that such a process can’t work, because feedback is required.

    -

    1 Craig Larman, “Agile and Iterative Development: A Manager’s Guide”, pp 102-107.

    +

    1 Craig Larman, “Agile and Iterative Development: A Manager’s Guide”, pp 102-107.

    Waterfall Model of Software Development

    @@ -7448,7 +7459,7 @@

    Authors: The Agile Manifesto

    Pre-Internet Internet Big Data
    - + @@ -7472,7 +7483,7 @@

    Authors: The Agile Manifesto

    - + @@ -7483,8 +7494,8 @@

    Authors: The Agile Manifesto

    Name Agile “School”1 Agile “School”1
    Kent Beck
    Martin Fowler Consultant2 Consultant2
    James Grenning Adaptive Software Development
    -

    1 The “named” Agile methodology this person is most associated with, at least at the time the manifesto was drafted.
    -2 Not involved in the invention of any particular “named” Agile methodology.

    +

    1 The “named” Agile methodology this person is most associated with, at least at the time the manifesto was drafted.
    +2 Not involved in the invention of any particular “named” Agile methodology.

    @@ -7493,11 +7504,11 @@

    Authors: The Agile Manifesto (cont.)

    - + - + @@ -7529,23 +7540,22 @@

    Authors: The Agile Manifesto (cont.)

    - +
    Name Agile “School”1 Agile “School”1
    Andrew Hunt Consultant, co-owner of the Pragmatic Programmers2 Consultant, co-owner of the Pragmatic Programmers2
    Ron Jeffries
    Dave Thomas Consultant, co-owner of the Pragmatic Programmers2 Consultant, co-owner of the Pragmatic Programmers2
    -

    1 The “named” Agile methodology this person is most associated with, at least at the time the manifesto was drafted.
    -2 Not involved in the invention of any particular “named” Agile methodology.

    +

    1 The “named” Agile methodology this person is most associated with, at least at the time the manifesto was drafted.
    +2 Not involved in the invention of any particular “named” Agile methodology.

    -

    Three Agile Methods: Scrum, XP, and Lean
    -
    -While there are several different agile methods mentioned, we’ll discuss the three most popular.1

    +

    Three Agile Methods: Scrum, XP, and Lean

    +

    While there are several different agile methods mentioned, we’ll discuss the three most popular.1

    • Scrum – focuses on project management.
    • XP – a complete, small-team approach, addressing project management and developer practices.
    • Lean – inspired by Toyota Lean, “Just-in-Time” Manufacturing.
    -

    1 See Highsmith’s “Agile Software Development Ecosystems” for an excellent survey of the major agile methods (in 2002).

    +

    1 See Highsmith’s “Agile Software Development Ecosystems” for an excellent survey of the major agile methods (in 2002).

    Three Agile Methods: Scrum, XP, and Lean

    @@ -7563,7 +7573,7 @@

    Scrum’s Process

    -

    (source: Wikipedia))

    +

    (source: Wikipedia)

    @@ -7883,7 +7893,7 @@

    Reading Assignment

    Reading Assignment (cont.)

    Last week, I had you read some general introductions to Agile. This week, I want you read about the three main Agile methodologies that we discussed today.

    @@ -7904,7 +7914,7 @@

    Mini-Project


    -

    Lecture 13: Complexity vs. Simplicity Part III: Simple Systems

    +

    Lecture 13: Complexity vs. Simplicity Part II: Simple Systems

    First, we complete last week’s content on Agile software methods. Then we finish the course lectures with a discussion of the importance of simplicity in software systems.

    @@ -8171,20 +8181,19 @@

    HTTP

    HTTP Protocol

    HTTP supports only 9 request methods (a.k.a. “verbs”): HEAD, GET, POST, PUT, DELETE, TRACE, and OPTIONS. Just GET and POST are used to do the vast majority of work on the Interwebs…

    -

    These nine methods let us do everything1 we’ve been doing on the web for the last ~15 years!

    -

    1 I mean the HTTP-based world-wide web, as it was originally conceived, not including other Internet protocols that we also use, like those for email.

    +

    These nine methods let us do everything1 we’ve been doing on the web for the last ~15 years!

    +

    1 I mean the HTTP-based world-wide web, as it was originally conceived, not including other Internet protocols that we also use, like those for email.

    HTTP Protocol

    -

    HTTP request messages consist of a list of headers (key-value pairs), one per line, followed by a blank line, followed by the message body (which may be empty).
    -
    -Here’s the output when requesting the Wikipedia page for HTTP, using curl1 and its --include option to also print the headers:

    +

    HTTP request messages consist of a list of headers (key-value pairs), one per line, followed by a blank line, followed by the message body (which may be empty).

    +

    Here’s the output when requesting the Wikipedia page for HTTP, using curl1 and its --include option to also print the headers:

     
     curl --include http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
     
    -

    1 wget is similar program available on many systems.

    +

    1 wget is similar program available on many systems.

    HTTP Protocol

    @@ -8353,7 +8362,20 @@

    Other Important Dates (2/2)

    I have decided to give you a take-home, open-book final exam. I will hand it out at the end of class next week and it will be due at the beginning of class on 12/14. (I’ll be in our classroom at the usual time.)

    -

    Lecture 14: Student Presentations on the Code Project and Course Review

    +

    Lecture 14: Student Presentations on “Mini” Projects

    +
    +
    +

    Mini-Project Presentations

    +

    Each of you does a 15 minute presentation, with questions, on your mini project.

    +
    +
    +

    Final Important Dates

    +
      +
    • 12/10 (Friday): Last day to turn in any other remaining assignments (by Midnight…).
    • +
    • 12/14: Final Exam Due
    • +
    +

    I will hand out the take-home, open-book final exam at the end of class today. It will be due at the beginning of class on 12/14. (I’ll be in our classroom at the usual time.)

    +

    If you’re running late to class, call or send me email.

    References

    @@ -8456,7 +8478,7 @@

    ACID, CAP, and My Problem with CAP and Yahoo’s Little Known NoSQL System.
  • Werner Vogels, Eventually Consistent. (Also here) A specific look at different kinds of eventual consistency and how they informed the design of several Amazon technologies.
  • Haifeng Yu and Amin Vahdat, Design and Evaluation of a Continuous Consistency Model for Replicated Services
  • -
  • Guy Pardon, >CAP Solution. An argument that if you don’t require all three properties at the same time, you can work around the CAP theorem.
    +
  • Guy Pardon, A CAP Solution. An argument that if you don’t require all three properties at the same time, you can work around the CAP theorem.
  • @@ -8671,11 +8693,10 @@

    Object-Oriented Programming (Specifics)

  • Singleton Design Pattern (Wikipedia).
  • Visitor Pattern (Wikipedia). A truly ugly pattern. A good illustration of forcing a “pure” OO solution when a much more elegant approach should be used instead, i.e., type classes (see the FP section below.)
  • Are Design Patterns Missing Language Features. A starting place for criticisms of patterns.
  • -
  • Peter Norvig, Design Patterns. Argues that 16 of the 23 GoF patterns are unnecessary in languages like Lisp.
    -
    -
  • +
  • Peter Norvig, Design Patterns. Argues that 16 of the 23 GoF patterns are unnecessary in languages like Lisp.
  • +

    Object-Oriented Programming (Specifics)

    @@ -8785,7 +8806,7 @@

    Scrum

    • Ken Schwaber, “Agile Project Management with Scrum”, Microsoft Press, 2004. ISBN 978-0-735-61993-7.
    • Ken Schwaber, “Agile Project Management with Scrum”, Microsoft Press, 2004. ISBN 978-0-735-61993-7.
    • -
    • Scrum Development) (Wikipedia).
      +
    • Scrum Development (Wikipedia).
    @@ -8858,7 +8879,8 @@

    Off Topic: Career Advice

    You might find useful…



    diff --git a/PragmaticsOfIndustrialSWDevelopment/html/lecture1.html b/PragmaticsOfIndustrialSWDevelopment/html/lecture1.html index 8f27aad..f250a64 100644 --- a/PragmaticsOfIndustrialSWDevelopment/html/lecture1.html +++ b/PragmaticsOfIndustrialSWDevelopment/html/lecture1.html @@ -106,13 +106,29 @@

    Course Administrivia

    -

    Schedule

    -

    (Revised 10/26)
    -| 8/31 | Course Intro and Scaling Computation, Part I (Multithreaded Programming, Actor Concurrency) |
    -| 9/07 | Scaling Computation, Part II (Agents, Software Transactional Memory, Memory is the New Disk) |
    -| 9/14 | Scaling Systems, Part I (Vertical vs. Horizontal Scaling, Distributed Systems like HTTP, etc.) Last day to withdraw without a W grade. |
    -| 9/21 | Scaling Data, Part I (SQL vs. NoSQL databases) |
    -| 9/28 | Scaling Data, Part II (Example NoSQL databases) |

    +

    Schedule (Revised 10/26)

    + + + + + + + + + + + + + + + + + + + + + +
    8/31 Course Intro and Scaling Computation, Part I (Multithreaded Programming, Actor Concurrency)
    9/07 Scaling Computation, Part II (Agents, Software Transactional Memory, Memory is the New Disk)
    9/14 Scaling Systems, Part I (Vertical vs. Horizontal Scaling, Distributed Systems like HTTP, etc.) Last day to withdraw without a W grade.
    9/21 Scaling Data, Part I (SQL vs. NoSQL databases)
    9/28 Scaling Data, Part II (Example NoSQL databases)

    Schedule (cont.)

    @@ -160,7 +176,7 @@

    Schedule (cont.)

    12/07 - Student Presentations on the Code Project, Course Review + Student Presentations on the Code Project 12/14 @@ -195,19 +211,19 @@

    Course Components: Readings and Lecture

    Course Components: Project Exercises

      -
    • Start with a “running” 3-tier Scala application
      - * Hosted on GitHub.com at github.com/deanwampler/AkkaWebSampleExercise.
      - * Uses actors to scale computation over a NYSE data set.
      - * Add features during the semester.
      - * Refactor the code to improve it and accommodate new features.
    • +
    • Start with a “running” 3-tier Scala application
    • +
    • Hosted on GitHub.com at github.com/deanwampler/AkkaWebSampleExercise.
    • +
    • Uses actors to scale computation over a NYSE data set.
    • +
    • Add features during the semester.
    • +
    • Refactor the code to improve it and accommodate new features.

    Course Components: Project Exercises (cont.)

      -
    • You are encouraged to work together.
      - * Like you would in an industrial setting…
      - * I expect the solutions to look “similar”.
    • +
    • You are encouraged to work together.
    • +
    • Like you would in an industrial setting…
    • +
    • I expect the solutions to look “similar”.
    @@ -217,21 +233,22 @@

    Course Components: Project Exercises (cont.)

  • 1/2 the grade differences will be based on GitHub checkin logs and the quality of those checkins.
  • If you pair program, include both names in the commit comments!
  • -h1. Course Components: Project Exercises (cont.) +
    +
    +

    Course Components: Project Exercises (cont.)

    • 5% will be based on an oral presentation on 12/7.
    • 15 minutes or so, with time for questions and discussion.
    • -
    • Topics to be worked out with me.
      -
      -

    -
    +
  • Topics to be worked out with me.
  • +
    +

    Midterm and Final Exam

    • Will cover the reading, lecture material, and concepts learned in the exercises.
    • -
    • Final exam content weighted:
      - * 2/3 of the material since the midterm
      - * 1/3 material before the midterm.
    • +
    • Final exam content weighted:
    • +
    • 2/3 of the material since the midterm
    • +
    • 1/3 material before the midterm.
    • Midterm: 10/19 (first half of that evening’s session).
    • Final exam: 12/14 (usual class time).
    @@ -249,9 +266,9 @@

    Miscellaneous

    Scaling Computation, Part I

      -
    • Two models of concurrency compared:
      - * Multithreaded Programming.
      - * The Actor Model of concurrency.
    • +
    • Two models of concurrency compared:
    • +
    • Multithreaded Programming.
    • +
    • The Actor Model of concurrency.
    @@ -283,7 +300,7 @@

    Is This Safe?

    public int getNext() { return value++; } -}

    +}

    (credit: all Multithreading examples adapted from “Java Concurrency in Practice”)

    @@ -297,7 +314,7 @@

    This Is Safe

    public synchronized int getNext() { return value++; } -}

    +}

    code/SafeSequence.java

    Things Can Go Wrong

    @@ -329,7 +346,7 @@

    Deadlock Example

    synchronized (right) { synchronized (left) { doSomethingElse(); } } -}

    code/LeftRightDeadlock.java

    +}

    code/LeftRightDeadlock.java

    Other Deadlock Hazards

    @@ -351,11 +368,10 @@

    Livelock

    A thread is not blocked, but it can’t complete work because of trying operations always fails.


    -
    +
  • One scenario is two contenders with identical strategies for trying again, often colliding in the same way. If one is greedy, both may succeed.
  • +
    +

    The Importance of Immutability

    What about access to immutable state? All the synchronization issues go away. Immutable objects have these virtues:

    code/manager-actor.scala

    +

    code/manager-actor.scala

    Actors in Scala (Akka)

    @@ -440,7 +456,7 @@

    Starting an Akka Actor

    import se.scalablesolutions.akka.actor.Actor_ // Creating an actor (one way...) and starting val manager = Actor.actorOf[Manager] -manager.start

    code/manager-actor2.scala

    +manager.start

    code/manager-actor2.scala

    Actor vs. ActorRef

    @@ -460,7 +476,7 @@

    Sending a Message to an Akka Actor

    case class Unregister(actor: ActorRef) case object Stop case object Start -

    code/actor-messages.scala

    +

    code/actor-messages.scala

    An Asynchronous (Fire and Forget) Message

    @@ -473,7 +489,7 @@

    An Asynchronous (Fire and Forget) Message

    case Start => managedActors foreach (_ ! Start) } } -

    code/actor-asynch-message.scala

    +

    code/actor-asynch-message.scala

    If a response is required, the receiver can send a message to the sender.

    @@ -490,7 +506,7 @@

    An Synchronous (Wait) Message

    } } } -

    code/actor-synch-message.scala

    +

    code/actor-synch-message.scala

    An Synchronous (Wait) Message (cont.)

    @@ -513,7 +529,7 @@

    An Asynchronous with a Future Message

    case None => "Error!" } } -

    code/actor-future-message.scala

    +

    code/actor-future-message.scala

    An Asynchronous with a Future Message (cont.)

    @@ -537,7 +553,7 @@

    Another Way to Write the Last Example

    case None => "Error!" } } -

    code/actor-future-message2.scala

    +

    code/actor-future-message2.scala

    Achieving Immutability

    @@ -697,9 +713,8 @@

    Add New Functionality (cont.)

    Add New Functionality (cont – sort of…)

    How could we modify filter to filter by instruments? (To make it easier, assume there is only one instrument in the list, as in the test.)

    This is not easy, as we really need to parse the JSON into some representation, filter out the records (which are of the form {"timestamp": 1283233404995, "symbol": "A", "price", "40.0"}) using the symbol, then convert back to a string.

    -

    Can you figure out how to do this with the Lift JSON APIs used in the application and JSON helper types provided, or using other 3rd-party JSON libraries?
    -
    -Actually, the right way to implement this feature is to support this filtering at the database level, which we will learn how to do with MongoDB.

    +

    Can you figure out how to do this with the Lift JSON APIs used in the application and JSON helper types provided, or using other 3rd-party JSON libraries?

    +

    Actually, the right way to implement this feature is to support this filtering at the database level, which we will learn how to do with MongoDB.

    diff --git a/PragmaticsOfIndustrialSWDevelopment/html/lecture13.html b/PragmaticsOfIndustrialSWDevelopment/html/lecture13.html index a40cb4b..b5b3ed3 100644 --- a/PragmaticsOfIndustrialSWDevelopment/html/lecture13.html +++ b/PragmaticsOfIndustrialSWDevelopment/html/lecture13.html @@ -49,7 +49,7 @@

    August 5, 2010

    -

    Lecture 13: Complexity vs. Simplicity Part III: Simple Systems

    +

    Lecture 13: Complexity vs. Simplicity Part II: Simple Systems

    First, we complete last week’s content on Agile software methods. Then we finish the course lectures with a discussion of the importance of simplicity in software systems.

    diff --git a/PragmaticsOfIndustrialSWDevelopment/html/lecture14.html b/PragmaticsOfIndustrialSWDevelopment/html/lecture14.html index 78c135a..ced9f72 100644 --- a/PragmaticsOfIndustrialSWDevelopment/html/lecture14.html +++ b/PragmaticsOfIndustrialSWDevelopment/html/lecture14.html @@ -49,7 +49,20 @@

    August 5, 2010

    -

    Lecture 14: Student Presentations on the Code Project and Course Review

    +

    Lecture 14: Student Presentations on “Mini” Projects

    +
    +
    +

    Mini-Project Presentations

    +

    Each of you does a 15 minute presentation, with questions, on your mini project.

    +
    +
    +

    Final Important Dates

    +
      +
    • 12/10 (Friday): Last day to turn in any other remaining assignments (by Midnight…).
    • +
    • 12/14: Final Exam Due
    • +
    +

    I will hand out the take-home, open-book final exam at the end of class today. It will be due at the beginning of class on 12/14. (I’ll be in our classroom at the usual time.)

    +

    If you’re running late to class, call or send me email.

    diff --git a/PragmaticsOfIndustrialSWDevelopment/html/references.html b/PragmaticsOfIndustrialSWDevelopment/html/references.html index 8ef3ffa..76c73ca 100644 --- a/PragmaticsOfIndustrialSWDevelopment/html/references.html +++ b/PragmaticsOfIndustrialSWDevelopment/html/references.html @@ -550,7 +550,8 @@

    Off Topic: Career Advice

    You might find useful…

    diff --git a/PragmaticsOfIndustrialSWDevelopment/slides/lecture1.slides b/PragmaticsOfIndustrialSWDevelopment/slides/lecture1.slides index d5c1505..6085464 100644 --- a/PragmaticsOfIndustrialSWDevelopment/slides/lecture1.slides +++ b/PragmaticsOfIndustrialSWDevelopment/slides/lecture1.slides @@ -37,9 +37,8 @@ h1. Course _Administrivia_ | twitter: | "@deanwampler":http://twitter.com/deanwampler | | office hours: | Tuesday, 6:45 - ? and by appointment | -h1. Schedule +h1. Schedule (Revised 10/26) -(Revised 10/26) | 8/31 | Course Intro and Scaling Computation, Part I (Multithreaded Programming, Actor Concurrency) | | 9/07 | Scaling Computation, Part II (Agents, Software Transactional Memory, Memory is the New Disk) | | 9/14 | Scaling Systems, Part I (Vertical vs. Horizontal Scaling, Distributed Systems like HTTP, etc.) _Last day to withdraw without a W grade._ | @@ -60,7 +59,7 @@ h1. Schedule (cont.) | 11/23 | Complexity vs. Simplicity Part I: Effective Software Development Processes, including Scrum, XP, and Lean | | 11/30 | Complexity vs. Simplicity Part II: Simple Systems | -| 12/07 | Student Presentations on the Code Project, Course Review | +| 12/07 | Student Presentations on the Code Project | | 12/14 | Final Exam (regular class time) | h1. Readings @@ -107,7 +106,7 @@ h1. Course Components: Project Exercises (cont.) * 1/2 the grade differences will be based on GitHub checkin logs and the quality of those checkins. * *If you _pair program_, include _both_ names in the commit comments!* - h1. Course Components: Project Exercises (cont.) +h1. Course Components: Project Exercises (cont.) * 5% will be based on an *oral presentation* on 12/7. * 15 minutes or so, with time for questions and discussion. diff --git a/PragmaticsOfIndustrialSWDevelopment/slides/lecture14.slides b/PragmaticsOfIndustrialSWDevelopment/slides/lecture14.slides index 71b5310..9ca9de3 100644 --- a/PragmaticsOfIndustrialSWDevelopment/slides/lecture14.slides +++ b/PragmaticsOfIndustrialSWDevelopment/slides/lecture14.slides @@ -1 +1,14 @@ -h1. Lecture 14: Student Presentations on the Code Project and Course Review +h1. Lecture 14: Student Presentations on "Mini" Projects + +h1. Mini-Project Presentations + +Each of you does a 15 minute presentation, with questions, on your mini project. + +h1. Final Important Dates + +* _12/10 (Friday)_: *Last* day to turn in any other remaining assignments (by Midnight...). +* _12/14_: Final Exam Due + +I will hand out the take-home, open-book final exam at the end of class today. It will be due at the _beginning of class on 12/14_. (I'll be in our classroom at the usual time.) + +If you're running late to class, call or send me email.