First International Workshop on
Reliable Data Services and Systems (RDSS)

Formerly Workshop on Testing Database Systems (DBTest)

Co-located with ACM SIGMOD 2014

Workshop Program

10:15 - 10:20 RDSS 2014 Welcome

10:20 - 11:30 RDSS Keynote

  • Data Systems Quality Deserves a Data-Driven approach
    • Seth Eliot, Principal Knowledge Engineer, Microsoft


Data systems are evolving.  Storage is distributed and so is processing.  Massive parallelization enables us to crunch increasing data sizes reaching past 100s of petabytes.  Many data systems are moving to cloud services, and even traditional database management systems are taking on the best aspects of services such as access to telemetry and system updates. How we assess the quality of our data management systems/services also needs to evolve.  Data-driven quality (DDQ) is the key to this evolution.

Testers have traditionally relied on test results, but the data sources now available to the tester and the ability to process these are expanding like never before.  DDQ strategies such as Testing in Production (TiP) are essential tools for most testers, but the question is how do teams implement these strategies to benefit their specific product?  The data needed can be big or small, real-time or delayed, synthetic or organic.  So it is critical that you know what data you need and understand how much is required. In this talk, you will learn about the latest tools and techniques used at Microsoft to get insights from this data about product quality.  Seth will take you through the critical thinking and framing to understand how data-driven quality can benefit your product and team.  He will then describe a roadmap for how teams can implement a data-driven quality strategy and take their testing to the next level.

Speaker's Bio:

Seth Eliot is Principal Knowledge Engineer for Microsoft Test Excellence focusing on driving best practices for services and cloud development/testing across the company.  He previously was Senior Test Manager, most recently for the team solving exabyte storage and data processing challenges for Bing, and before that enabling developers to innovate by testing new ideas quickly with users “in production” with the Microsoft Experimentation Platform. Testing in Production (TiP), software processes, cloud computing, and other topics are ruminated upon at Seth's blog at and on Twitter (@setheliot). Prior to Microsoft, Seth applied his experience at delivering high quality software services at where he led the Digital QA team to release Amazon MP3 download, Amazon Instant Video Streaming, and Kindle Services.


11:30 - 12:00 Paper Session I

  •  Continuous performance testing for SAP HANA
    • Alexander Boehm (SAP AG), Kim-Thomas Rehmann (SAP AG), Alexander Boehm (SAP AG), Dong Hun Lee (SAP AG), Joerg Wiemers (SAP AG)

12:00 - 13:45 Lunch break

13:45 - 14:45 Paper Session II

  • Extensions of BG for Testing and Benchmarking Alternative Implementations of Feed Following
    • Sumita Barahmand (USC), Shahram Ghandeharizadeh (USC), Dobromir Montauk (Google, Inc)
  • Towards the Implementation of Bug-Free Database Applications: A Formal Approach
    • Iman Saleh (University of Miami), Gregory Kulczycki (Virginia Tech), M. Brian Blake (University of Miami)

14:45 - 15:00 Break

15:00 - 16:00 Paper Session III

  • Deployment-Aware Index Tuning
    • Quoc Trung Tran (NUS),Ivo Jimenez, (UC Santa Cruz), Rui Wang (UC Santa Cruz), Neoklis Polyzotis (UC Santa Cruz)
  • An empirical analysis of database recovery costs
    • Caetano Sauer (University of Kaiserslautern), Goetz Graefe (HP Labs), Theo Härder (University of Kaierslautern)

16:10 - 16:15 Closing Remarks

Invited Talk <cancelled>

  • Building reliable large-scale distributed systems at scale
    • Khawaja Shams, Amazon Web Services, NoSQL


Amazon’s Dynamo paper started a revolution in distributed systems and the world of NoSQL databases. The contributions from this paper are still impacting the design and practices of some of the world’s largest distributed systems, including those at and beyond.

We will go over the journey from relational databases to Dynamo to DynamoDB and what we learned along the way.

While the distributed systems at AWS are built on core principles of distributed system design for consensus, verification, and scaling, translating theory into real production systems requires serious engineering effort and some key innovations. We will discuss the bruises we proudly earned along the way and the instrumental real world lessons that helped us transform theoretically sound systems into production services like DynamoDB. These insights include techniques for mitigating failures by limiting the blast radius of individual system components, leveraging eventual consistency while also applying consistency wherever it is appropriate, simulating failures (such as network partitions or datacenter failures) that have been proven in practice, and how we’ve used core distributed systems fundamentals to build highly scalable, performant, durable, and resilient systems. We’ll share the fundamental tenets that define DynamoDB, SQS, and SNS as truly Internet-scale services.

Speaker's bio:

Khawaja Shams, Head of Engineering for NoSQL at Amazon Web Services. He is a passionate distributed systems engineer, loves to code, and all it takes to distract him is a refreshing discussion around coding a new distributed systems app. Together, Swami and Khawaja are on a mission to redefine NoSQL, again! Khawaja also serves as the Technical Assistant to Charlie Bell, VP of Utility Computing at AWS. Prior to joining Amazon, Khawaja led the Data Services team at NASA Jet Propulsion Laboratory, where he owned distributed systems for tactical operations of dozens of missions across NASA including the Curiosity Mars rover, Voyager, Deep Space Network, Cassini, and International Space Station. Khawaja was one of the key leaders influencing cloud adoption across NASA and advised multiple government agencies and enterprises on Cloud Computing. Khawaja owned development of the data processing pipelines for all Mars images as they are beamed backed to earth and pushed out to global community of scientist, operators, as well as the public.