Performance for Product Developers

Oct 12, 20122 likes1,440 views

This document discusses performance optimization for product developers building add-ons for Plone. It recommends using tools like plone.app.caching, Dexterity instead of Archetypes, and OOBTree for data structures to improve performance. It emphasizes testing caching functionality and providing rulesets to make caching easier for integrators. The presenter offers help from the Code Distillery in implementing these techniques.

Performance for Product Developers
Matthew Wilkes
Friday, 12 October 12 1

Who am I?

• Performance and security work
at the Code Distillery
• Framework and security teams,
membership committee
• Foundation board member

Friday, 12 October 12 2

Who am I?

• Zope / Plone core developer
• Silly accent. Sorry.

• Not a morning person. Sorry.

Friday, 12 October 12 3

Performance…

• Plone is pretty damn fast
• Low edit sites are really easy
to make ﬂy
• Runs ﬁne on a 256mb VM

Friday, 12 October 12 4

Performance…

• plone.app.caching takes about
5 minutes to conﬁgure
• Add Varnish and it goes really
rather quick
• Writing is harder

Friday, 12 October 12 5

… for Product Developers?

Friday, 12 October 12 7

… for Product Developers

• Security isn’t the sole
responsibility of the integrator
• Neither is performance
• Everyone is careful about the
code they write

Friday, 12 October 12 8

some even over do it
Friday, 12 October 12 9

Recap

• Don’t add things to the
catalog if you can help it
• If you have to, use an indexer,
not just a raw FieldIndex

Friday, 12 October 12 10

Recap

• Build your types on
Dexterity, not Archetypes
• If you’re not sure what
structure to use for your
data, the answer is OOBTree

Friday, 12 October 12 11

Recap

• Avoid big object churn by
keeping Lengths of things
you need
• Find a balance between huge
objects and annotation soup

Friday, 12 October 12 12

Recap

• If you see for loops inside
for loops you’re probably
missing a simpler way

• Don’t forget sets and
itertools, they’re good

Friday, 12 October 12 13

… for Product Developers

• Very few people are using the
tools we have available
• Equally true for publicly available
products as for policies

Friday, 12 October 12 14

Who’s used this?
Friday, 12 October 12 15

You guys! Seriously‽
Friday, 12 October 12 16

Access time for a browser view through Varnish
80ms
23

60ms
56 processing
connect
40ms

20ms

0ms 0
without cache:ruleset with cache:ruleset

I love this chart
Friday, 12 October 12 17

plone.app.caching

• Assign caching rules directly to contexts by
interface

• Can map to an operation, like
plone.app.caching.strongCaching

• or to a ruleset, like
plone.content.itemView

Friday, 12 October 12 18

psst, integrators!
• You can have commit access
today
• If your favourite product doesn’t
have rulesets deﬁned, add them
• I’ll bring “I cleaned up your mess”
stickers to Brasília
Friday, 12 October 12 19

Edge Side Includes

photo (c) http://www.ﬂickr.com/photos/fornal/
Friday, 12 October 12 20

ESIs

• Varnish is your friend
• Lets integrators do page
composition
• but easy to leak data

Friday, 12 October 12 21

ESI

• Suddenly, cache hits for
logged in users!
• aaaand, cache hits on private
pages for anonymous
• Really rather bad
Friday, 12 October 12 23

ESI

• Set ETags to
roles|lastModified?

• ETags can be faked
• Still leaks data
Friday, 12 October 12 24

ESI

• Integrators will still have to
segment their cache manually
• Hard work
• A lot easier if add-ons are
written with this in mind

Friday, 12 October 12 25

ESI

• Plan your templates for eventual
ESIs
• Many small templates is
appropriate, lets them be
customised

Friday, 12 October 12 26

ESI
• If there is a block of markup that
is mostly static but uses the user
in some bit, make sure that’s
separated out
• Integrators shouldn’t need more
than a few minutes to add ESI to
your templates
Friday, 12 October 12 27

Testing caching

photo (c) http://www.ﬂickr.com/photos/alisdair/
Friday, 12 October 12 28

Testing

• Far too few people run
automated tests on their
caching
• Well, far too few people run
automated tests at all

Friday, 12 October 12 29

Testing

• If you have plone.app.testing
based tests already it’s easyish
• Create a new layer using a
ZServer and start up Varnish
• Make some testbrowser calls

Friday, 12 October 12 30

Things that need building

Friday, 12 October 12 31

Utility packages

• We need helper classes and
methods for writing caching tests
• Actual tests in popular packages,
as examples

Friday, 12 October 12 32

Rulesets

• The current plone.app.caching
rulesets are integrator focused
• We need some standard ones
for product developers to use

Friday, 12 October 12 33

The Code Distillery
Bristol

Questions?

Want help putting this into practice?
sales@thedistillery.eu
Friday, 12 October 12 34

The document discusses the benefits of distributed version control systems (DVCS) like Git and Mercurial compared to centralized version control systems like Subversion. It describes how Atlassian transitioned their development teams from Subversion to DVCS. Their approach included providing good tooling, linking commits to issue trackers, bringing over history from Subversion, appointing mentors, and emphasizing practices like committing early and often. The benefits seen at Atlassian included increased developer speed, faster feedback, and improved collaboration.

垂直社区的产品改造Dahui Feng

This document outlines a plan to redesign and improve the products of a vertical community (垂直社区) over multiple phases. It discusses analyzing the current problems with the community's products, performing initial monitoring, tuning and stabilizing work in Phase I. Phase II focuses on improving information architecture and code. Later phases cover user experience, new features, security and ongoing improvements through a Plan-Do-Check-Adjust process. The goal is to transition the 12 year old community platform to better serve its 2.8 million users into the future.

static ABAP code analyzersMarkus Theilen

How Atlassian Makes Its Wiki StickyAtlassian

The document discusses how to make a wiki "sticky", which refers to keeping users engaged with the wiki through frequent visits and participation. It defines stickiness and presents a "sticky web matrix" to measure how sticky a wiki is. It then provides suggestions on how to increase stickiness, such as giving the wiki a friendly name, branding it, encouraging content creation, making content easy to find, using notifications and recommendations, and measuring and rewarding participation. The overall goal is to make the wiki a central place for users to easily access and share important information.

Stepping into Usable WebShajed Evan

iFixit @ Monterey Bay AquariumKyle Wiens

The document discusses how repairing and reusing existing things is more efficient and cost-effective than constantly mining for new raw materials. It notes that the Earth has limited resources and we cannot rely on a linear manufacturing process indefinitely. Repairing things is often free or cheaper than replacing them, and taking things apart is a good way to understand how they work. Self-repair fosters sustainability and connections between people and devices.

Open Source Security – A vendor's perspectiveMatthew Wilkes

Progressive Enhancement using WSGIMatthew Wilkes

This document discusses WSGI (Web Server Gateway Interface), a Python standard for building web applications and APIs. WSGI allows web applications to be composed together through middleware. Middleware can modify requests and responses to add features like theming, error handling, and URL rewriting. The document advocates for using middleware to progressively enhance applications with features like CAPTCHAs in a reusable way across multiple frameworks and projects. An example is given of a WSGI middleware that inserts CAPTCHAs into forms on the fly without requiring changes to the underlying applications.

[Phind] MiracleChia-Yu Kuo

The document describes a proposed solution called "Miracle" to solve the problem of files being separated across multiple computers. Miracle involves a pair of USB drives that would sync files in real-time between devices, providing an alternative to cloud storage services like Dropbox that have issues with cost, storage limits, sync times, and privacy/security. The inventor believes there is a large potential market and that the proprietary peer-to-peer syncing technology could be sold or licensed to other companies. However, he acknowledges lacking expertise in areas like marketing, fundraising, and recruiting a team.

Things I hate to do as a designer but do anywayMatt Johnston

[JVMLS 12] Kotlin / Java InteropAndrey Breslav

This document discusses Kotlin and its interoperability with Java. It outlines Kotlin's features like its smart compiler, static typing, and tool support. It then discusses how Kotlin enhances existing Java APIs through extensions and collections interfaces. It also describes Kotlin's approaches to preventing null pointer exceptions through nullable types and annotations. Finally, it examines how Kotlin handles static concepts like packages and class objects that don't exist directly in Kotlin.

Optimizing WordPress Performance on Shared Web HostingJon Brown

This document discusses performance tweaks that can be made for WordPress sites on shared hosting. It is divided into three acts: inside WordPress, on the shared server, and off the shared server. Inside WordPress, it recommends right-sizing images, checking for 404 errors, keeping the database under control, and using caching plugins. On the shared server, it suggests updating to newer PHP versions, cleaning up the database, and using CloudFlare. Off the server, it only recommends using CDNs like CloudFlare for their free benefits.

Why not to use Rails? (actually it's when not to use Rails)Arik Fraimovich

The document discusses when to use and not use Ruby on Rails frameworks for web applications. Some key benefits of Rails include its large community, simplicity, and convention over configuration approach. However, it notes that Rails can become slow for large applications or non-web applications. It suggests using Rails carefully as projects grow and considering other options like actor models, JRuby, or TorqueBox that allow for more concurrency. The document wraps up by saying to use Ruby and Rails judiciously while remembering there are other tools available.

Introduction to the DisruptorTrisha Gee

The document discusses the Disruptor, a data structure and work flow that allows for high-performance concurrent programming with no contention. The Disruptor uses a ring buffer to pass messages between threads very quickly in a parallel manner. Publishers can insert events into the ring buffer, while batch event processors can read the events in batches to process them in parallel threads. The Disruptor framework encourages modeling the problem domain and provides reliable ordering, parallelism, and high performance.

MySQL Cluster no PayPalMySQL Brasil

Daniel Austin of PayPal presented on using MySQL Cluster to build a globally distributed database called YESQL. He discussed common myths about big data and NoSQL databases, including that big data always requires NoSQL and that the CAP theorem is simpler than it really is. Austin explained how MySQL Cluster was used to build YESQL to meet requirements like high availability, scalability and global replication within 1000ms. He reviewed the architecture involving tiling across AWS availability zones and lessons learned.

OmniOS Motivation and Design ~ LISA 2012Theo Schlossnagle

1. The document discusses the motivation and goals in building an Illumos-based OS called OmniOS. The key goals were to have ABI stability, ZFS, zones, DTrace, and be open source while also being installable via network and having consistent multi-architecture support. 2. The document outlines the release cycles established with regular minor weekly updates and major releases every 6 months, as well as goals around commercial support. 3. OmniOS is designed to be minimal, acting as a base for others to build more comprehensive distributions on top of, and is available via various methods like Vagrant boxes, ISOs, and AMIs.

Java performance: What's the big deal? - Trisha GeeJAX London

It seems that everywhere you look these days people are needing to work on high performance/low latency applications. If they're not already working on them, then they want to be. But what do you really need to know when you're working on performance-sensitive systems? Where do you get started? In this session, Trisha outlines things you need to consider when thinking about performance, covers some of the frequently-asked questions, and notes the pitfalls and common traps that developers fall into.

Enterprise javascriptsession1Troy Miles

Iwmn architectureLenz Gschwendtner

The document discusses the evolution of a Catalyst application to use lighter weight components including removing the model layer and using Redis for sessions and RabbitMQ for the backend. It started as a standard Catalyst app and was stripped down over time, moving content to CouchDB, sessions to Redis, and the backend to RabbitMQ. Lessons learned included using plugins, choosing technologies like Redis, CouchDB and RabbitMQ, and getting a minimum viable product out quickly before refactoring.

2012 Q4 Cooperative Support for CAS UpdateAndrew Petro

GitHub Notable OSS Projectroumia

Adapt and respond: keeping responsive into the futureChris Mills

Media queries blah blah blah. You've all heard that talk a hundred times, so I won't do that. Instead, I'll go beyond the obvious, looking at what we can do today to adapt our front-ends to different browsing environments, from mobiles and other alternative devices to older browsers we may be called upon to support. You'll learn advanced media query and viewport tricks, including a look at @viewport, Insights into responsive images: problems, and current solutions, providing usable alternatives to older browsers with Modernizr and YepNope, other CSS3 responsive goodness - multi-col, Flexbox, and more, and finally where RWD is going ‚Äî matchMedia, CSS4 media queries, etc.

Dan node meetup_socket_talkIshi von Meier

This document summarizes Dan Getelman's presentation on using Node.js and Socket.io for real-time functionality at Lore. It discusses Lore's architecture using a front-end framework, Node.js, Socket.io for real-time communications, and a Python API. It describes how they ended up with this architecture to build the best experience for teachers and students. It also outlines how their system works with Redis for messaging, caching, and publishing updates, and how the front-end receives and handles messages.

Fast Mobile UIsWooga

Inside the Atlassian OnDemand Private CloudAtlassian

The document summarizes the development of Atlassian's private cloud platform. It describes how an initial team built a secret test environment (Block-1) and then a larger test environment (Block-2) to validate the architecture. Over time, the platform grew to 13,500 VMs which led to issues like poor performance and slow deployments. The team then focused on optimizing the platform infrastructure using technologies like OpenVZ containers to reduce overhead and read-only OS images to improve consistency and simplify management. The summaries emphasize how the team took an iterative approach, testing concepts on small scales before full deployment to address issues and focus on the tasks.

Big Data - architectural concerns for the new ageDebasish Ghosh

This document discusses big data architectural concerns for handling large volumes of data. It notes that new technologies have enabled more efficient use of big data, creating a positive feedback loop where more adoption of big data leads to even more data generation. This requires new ways of storing, retrieving, scaling and analyzing data in a distributed manner. Examples of distributed databases like Bigtable and Dynamo are discussed. The document recommends flexible schemas, storing data close to its domain model, and limiting data movement for efficiency.

MuraCon 2012 - Creating a Mura CMS plugin with FW/1jpanesar

Creating a Mura CMS Plugin with FW/1 involves the following steps: 1. Download the FW/1 base plugin template from the Mura Marketplace. 2. Configure the plugin by naming it and setting the app and package names. 3. Upload the configured plugin to Mura via the admin global settings plugins tab. 4. Develop the plugin functionality by adding controllers, views, and other files within the FW/1 folder structure and reloading Mura.

Make something real for Firefox OS with Mozilla app templatesPiotr Zalewa

The document discusses building apps for Firefox OS using Mozilla app templates. It describes emulating Firefox OS, available tools for app development including X-Tags building blocks, pre-app submission steps, and templates for a standard app stub, 2D game, and server app. Templates include necessary components and are aimed to allow building a basic app in a day. The document encourages attendees to use the templates to build their own awesome Firefox OS apps.

Performance for Product Developers

1. Performance for Product Developers Matthew Wilkes Friday, 12 October 12 1

2. Who am I? • Performance and security work at the Code Distillery • Framework and security teams, membership committee • Foundation board member Friday, 12 October 12 2

3. Who am I? • Zope / Plone core developer • Silly accent. Sorry. • Not a morning person. Sorry. Friday, 12 October 12 3

4. Performance… • Plone is pretty damn fast • Low edit sites are really easy to make ﬂy • Runs ﬁne on a 256mb VM Friday, 12 October 12 4

5. Performance… • plone.app.caching takes about 5 minutes to conﬁgure • Add Varnish and it goes really rather quick • Writing is harder Friday, 12 October 12 5

6. Performance… Friday, 12 October 12 6

7. … for Product Developers? Friday, 12 October 12 7

8. … for Product Developers • Security isn’t the sole responsibility of the integrator • Neither is performance • Everyone is careful about the code they write Friday, 12 October 12 8

9. some even over do it Friday, 12 October 12 9

10. Recap • Don’t add things to the catalog if you can help it • If you have to, use an indexer, not just a raw FieldIndex Friday, 12 October 12 10

11. Recap • Build your types on Dexterity, not Archetypes • If you’re not sure what structure to use for your data, the answer is OOBTree Friday, 12 October 12 11

12. Recap • Avoid big object churn by keeping Lengths of things you need • Find a balance between huge objects and annotation soup Friday, 12 October 12 12

13. Recap • If you see for loops inside for loops you’re probably missing a simpler way • Don’t forget sets and itertools, they’re good Friday, 12 October 12 13

14. … for Product Developers • Very few people are using the tools we have available • Equally true for publicly available products as for policies Friday, 12 October 12 14

15. Who’s used this? Friday, 12 October 12 15

16. You guys! Seriously‽ Friday, 12 October 12 16

17. Access time for a browser view through Varnish 80ms 23 60ms 56 processing connect 40ms 20ms 0ms 0 without cache:ruleset with cache:ruleset I love this chart Friday, 12 October 12 17

18. plone.app.caching • Assign caching rules directly to contexts by interface • Can map to an operation, like plone.app.caching.strongCaching • or to a ruleset, like plone.content.itemView Friday, 12 October 12 18

19. psst, integrators! • You can have commit access today • If your favourite product doesn’t have rulesets deﬁned, add them • I’ll bring “I cleaned up your mess” stickers to Brasília Friday, 12 October 12 19

20. Edge Side Includes photo (c) http://www.ﬂickr.com/photos/fornal/ Friday, 12 October 12 20

21. ESIs • Varnish is your friend • Lets integrators do page composition • but easy to leak data Friday, 12 October 12 21

22. Friday, 12 October 12 22

23. ESI • Suddenly, cache hits for logged in users! • aaaand, cache hits on private pages for anonymous • Really rather bad Friday, 12 October 12 23

24. ESI • Set ETags to roles|lastModified? • ETags can be faked • Still leaks data Friday, 12 October 12 24

25. ESI • Integrators will still have to segment their cache manually • Hard work • A lot easier if add-ons are written with this in mind Friday, 12 October 12 25

26. ESI • Plan your templates for eventual ESIs • Many small templates is appropriate, lets them be customised Friday, 12 October 12 26

27. ESI • If there is a block of markup that is mostly static but uses the user in some bit, make sure that’s separated out • Integrators shouldn’t need more than a few minutes to add ESI to your templates Friday, 12 October 12 27

28. Testing caching photo (c) http://www.ﬂickr.com/photos/alisdair/ Friday, 12 October 12 28

29. Testing • Far too few people run automated tests on their caching • Well, far too few people run automated tests at all Friday, 12 October 12 29

30. Testing • If you have plone.app.testing based tests already it’s easyish • Create a new layer using a ZServer and start up Varnish • Make some testbrowser calls Friday, 12 October 12 30

31. Things that need building Friday, 12 October 12 31

32. Utility packages • We need helper classes and methods for writing caching tests • Actual tests in popular packages, as examples Friday, 12 October 12 32

33. Rulesets • The current plone.app.caching rulesets are integrator focused • We need some standard ones for product developers to use Friday, 12 October 12 33

34. The Code Distillery Bristol Questions? Want help putting this into practice? sales@thedistillery.eu Friday, 12 October 12 34

Performance for Product Developers

Recommended

More Related Content

Similar to Performance for Product Developers (20)

Performance for Product Developers