Innovation without
      permission

              Daniel Lemire
http://lemire.me http://twitter.com/lemire


                      Thanks to: A. Badia, Louisville University and
                                J. Robillard from UQAM
- 2000 employees
 - 600 million users*



                                                                 * As of January 2011

Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
- No schema : key-value stores
 - No join
 - Engineers have direct access to data



                                                                 * As of January 2011

Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
~0%
 Image source: dullhunk
         (flickr)
~10 000 Information Systems
~90% Relational
100-200 Tables/database
50-200 Attributes/table

       Source: Brodie & Liu, The Power and Limits of Relational Technology in the Age of
            Information Ecosystems, On The Move Federated Conferences, 2010.
Post-Methodology Era:
late 1990s
  D. E. Avison and G. Fitzgerald, Where now for development methodologies? 2003.
Sophisticated users
     Image source: Dave77459
              (flickr)




                               Billions of computers
                                      Image source: ivanx
                                            (flickr)
Users are considered as mere
faceless objects for who the
systems are designed.



       J. Iivari, H. Isomäki, S. Pekkola, The user – the great unknown of systems development:
     reasons, forms, challenges, experiences and intellectual contributions of user involvement,
                                     Information Systems Journal, 2010.
93% of accounts are never used
      Source: Meredith and O'Donnell, A Functional Model of Social Media and its Application
                            to Business Intelligence, DSS '10, 2010.
n !ot
          s are never used
        a ted
    w ul
  I s
93% of accounts


   c o  n
     Source: Meredith and O'Donnell, A Functional Model of Social Media and its Application
                           to Business Intelligence, DSS '10, 2010.
Deployment: test for user reactions




                                                                 * As of January 2011

Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
- Google had more than
  1 million servers* in
          2007
 * according to Gartner
Brewer’s theorem (CAP)




          Consistency                Availability
                              XN
                       B MS         oS
                                        QL
                  RD



                    Tolerance




                   Gilbert, S. and Lynch, N., Brewer's conjecture and the feasibility of
                   consistent, available, partition-tolerant web services. 2002
NoSQL
- Corruption in Oracle database
  - Up to 16.5 million customers affected
  - $132 million frozen
  - thousands of loan applications lost

  - Over-engineered database: strong
  consistency throughout

Online: Chris Mellor, Morgan Chase blames Oracle for online bank crash ,
 Curt Monash, Details of the JPMorgan Chase Oracle database outage
Does your
methodology
know about:


      - Co-design with users?
      - Highly distributed data?

Innovation without permission: from Codd to NoSQL

  • 1.
    Innovation without permission Daniel Lemire http://lemire.me http://twitter.com/lemire Thanks to: A. Badia, Louisville University and J. Robillard from UQAM
  • 3.
    - 2000 employees - 600 million users* * As of January 2011 Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
  • 4.
    - No schema: key-value stores - No join - Engineers have direct access to data * As of January 2011 Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
  • 5.
    ~0% Image source:dullhunk (flickr)
  • 6.
    ~10 000 InformationSystems ~90% Relational 100-200 Tables/database 50-200 Attributes/table Source: Brodie & Liu, The Power and Limits of Relational Technology in the Age of Information Ecosystems, On The Move Federated Conferences, 2010.
  • 7.
    Post-Methodology Era: late 1990s D. E. Avison and G. Fitzgerald, Where now for development methodologies? 2003.
  • 8.
    Sophisticated users Image source: Dave77459 (flickr) Billions of computers Image source: ivanx (flickr)
  • 9.
    Users are consideredas mere faceless objects for who the systems are designed. J. Iivari, H. Isomäki, S. Pekkola, The user – the great unknown of systems development: reasons, forms, challenges, experiences and intellectual contributions of user involvement, Information Systems Journal, 2010.
  • 10.
    93% of accountsare never used Source: Meredith and O'Donnell, A Functional Model of Social Media and its Application to Business Intelligence, DSS '10, 2010.
  • 11.
    n !ot s are never used a ted w ul I s 93% of accounts c o n Source: Meredith and O'Donnell, A Functional Model of Social Media and its Application to Business Intelligence, DSS '10, 2010.
  • 13.
    Deployment: test foruser reactions * As of January 2011 Agarwal, A. (2009). Facebook: Science and the Social Graph. QCon 2008.
  • 14.
    - Google hadmore than 1 million servers* in 2007 * according to Gartner
  • 16.
    Brewer’s theorem (CAP) Consistency Availability XN B MS oS QL RD Tolerance Gilbert, S. and Lynch, N., Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. 2002
  • 17.
  • 18.
    - Corruption inOracle database - Up to 16.5 million customers affected - $132 million frozen - thousands of loan applications lost - Over-engineered database: strong consistency throughout Online: Chris Mellor, Morgan Chase blames Oracle for online bank crash , Curt Monash, Details of the JPMorgan Chase Oracle database outage
  • 19.
    Does your methodology know about: - Co-design with users? - Highly distributed data?

Editor's Notes

  • #2 \n
  • #3 \n
  • #4  dynamic redesign (new schemas) ex. twitter tag or re were not part of the system\n
  • #5  dynamic redesign (new schemas) ex. twitter tag or re were not part of the system\n
  • #6 \n
  • #7 \n
  • #8 \n Business Intelligence: 22% growth in 2008, over 8 billion$, Problem: I wasn’t consulted\n
  • #9 \n Business Intelligence: 22% growth in 2008, over 8 billion$, Problem: I wasn’t consulted\n
  • #10 \n Business Intelligence: 22% growth in 2008, over 8 billion$, Problem: I wasn’t consulted\n
  • #11 \n Business Intelligence: 22% growth in 2008, over 8 billion$, Problem: I wasn’t consulted\n
  • #12 \n Business Intelligence: 22% growth in 2008, over 8 billion$, Problem: I wasn’t consulted\n
  • #13  dynamic redesign (new schemas) ex. twitter tag or re were not part of the system\n
  • #14  dynamic redesign (new schemas) ex. twitter tag or re were not part of the system\n
  • #15 \n
  • #16  allow a small team with an idea to innovate quickly\n
  • #17  had the human population followed a similar growth there would be 55 trillion people on earth\n
  • #18  Tools are not neutral. Some encourage experimentation and flexibility, others do not.\n talk also about how nosql make dba less useful\n Tolerance for mistakes\n Tolerance for disagreements\n Tolerance for imprecisions \n but why can't you make your own out of open source parts?\n what's hard to get right? persistence (persistent ram) and concurrency (languages are getting better and easier) \n
  • #19  dynamic redesign (new schemas) ex. twitter tag or re were not part of the system\n
  • #20 \n