One of the blessings and curses of Rails development is the ability to use Rails console for debugging issues and inspecting data. The console is oftentimes used in a production environment, as it is the quickest method to glean information about any problems. With great power comes great responsibility: A command that attempts to reset a local developer environment by deleting all records of a model could easily be input into a production console. In addition, with potentially unknown clipboard data (thanks "copy-on-select"), any valid Ruby code with line breaks will automatically be executed if pasted. Any user who is deleting models, queuing up events, and updating records accidentally or intentionally should be made aware of the implications. At Salsify, we've solved this problem using a combination of open-source and home-brewed improvements and rolled them into a handy gem. Read on to learn more!
Have you ever wanted to build your own calculator, query language, or even web browser? Parsers and Transformers are tremendously useful for these applications and many more, and thanks to tools like Salsify's own Botanist you don't need to be an expert in compiler design to work with them.
Read More →
One of the most important core principles of developing with Ember is "data down, actions up." You might see this concept abbreviated as DDAU in various Ember communities. The main premise of DDAU is that data should flow down through your component hierarchy (passed through and potentially modified by various components), while changes to said data should be propagated back up through that hierarchy via actions.
Read More →
Read More →
But despite all of the advantages of running background jobs, under real world usage you can still run into challenging situations that require thoughtful handling. One general class of problems that can arise is achieving fairness in resource usage across users.
Read More →
Many web interfaces let a user effortlessly page through large sets of data. Implementing database queries that fetch these pages is also effortless for the programmer, usually requiring an OFFSET
and LIMIT
in the case of SQL and a FROM
and SIZE
in the case of Elasticsearch. Although this method is easy on the user and programmer, pagination queries of this type have a high hidden cost for SQL, Elasticsearch and other database engines.
At Salsify, we encountered this problem when implementing a feature to allow a user to step through records in a large, heavily filtered and sorted set. We had to implement an efficient pagination solution that would work in both our SQL and ES datastores. In this post we’ll look at an interesting technique to make pagination efficient for the database with large datasets. Specifically, we’ll look at implementation in SQL as well as how to translate this method to Elasticsearch.
Read More →
Have you ever noticed that no one writes "How we name ourRuby variables at Company X" blog posts? No one's making the Hacker News front page with "I combined these two strategies for method naming and suddenly my JavaScript is maintainable!" And yet when it comes to CSS, developers are all about naming strategies. We sing the praises of BEM, SMACSS, OOCSS, SUIT, or whatever other set of capital letters is popular this week. Why is that?
Read More →
Introduction
Using vim has been one of the greatest productivity boosters for my development life. I got into vim as a lowly system administrator because it seemed to be the tool of the trade. From there, my knowledge grew and now it is my editor of choice for almost all projects. This post will go through my current setup for Ruby on Rails development. I'd like to give a huge shout out to @tpope who has created a plethora of amazing plugins all worth making a part of your daily workflow.
Read More →
In Pursuit of a Scalable Ruby Offline Sort: Adventures in Ruby Memory Management
By Matthew Cross on Nov 5, 2015 8:15:00 AM
Here at Salsify, many of our customers regularly import large amounts of tabular product data into the system. This data needs to be sorted prior to being handled by different parts of the import process. Since we are running on Heroku, memory is a scarce resource. Sorting these arbitrarily large tabular data files requires great care. Reading all of the data into memory at once can result in extremely long execution times due to increased pressure on the Ruby garbage collector and can starve other processes on the same system of memory.
We needed a way to sort large files using a predictable amount of memory. Big data technologies like map reduce were overkill for our data scale. Creating an offline-sort gem to do this turned out to be quite the adventure and forced me to dig deeper into how Ruby manages memory, ultimately requiring a specialized heap implementation.
Read More →
At Salsify we run most of our Ruby on Rails based infrastructure in Heroku. We recently switched our Puma web workers over from Heroku 2X dynos to more powerful Heroku PX dynos and saw dramatic performance improvements: a 51% reduction in mean response time and a 59% reduction in 99th percentile response time. Based on this we did some experiments running our more memory/CPU intensive Delayed Jobs on PX dynos and saw a similarly encouraging 43% reduction in mean job execution time and 66% reduction in 99th percentile job execution time. This was great for a proof of concept, but only one of the eight cores on the PX dyno was being used. In order to take this to production in a cost-effective manner, we had to figure out how to utilize all of the cores on the dyno.