The Decision
We, like many small startups, had started our application on Heroku, and had built up considerable technical and social practice around its use. About 18 months ago, the Engineering team here at Salsify recognized that as our team, product, and user base continued to grow, we would outgrow the ideal case for Heroku.
One of the blessings and curses of Rails development is the ability to use Rails console for debugging issues and inspecting data. The console is oftentimes used in a production environment, as it is the quickest method to glean information about any problems. With great power comes great responsibility: A command that attempts to reset a local developer environment by deleting all records of a model could easily be input into a production console. In addition, with potentially unknown clipboard data (thanks "copy-on-select"), any valid Ruby code with line breaks will automatically be executed if pasted. Any user who is deleting models, queuing up events, and updating records accidentally or intentionally should be made aware of the implications. At Salsify, we've solved this problem using a combination of open-source and home-brewed improvements and rolled them into a handy gem. Read on to learn more!
Have you ever wanted to build your own calculator, query language, or even web browser? Parsers and Transformers are tremendously useful for these applications and many more, and thanks to tools like Salsify's own Botanist you don't need to be an expert in compiler design to work with them.
One of the most important core principles of developing with Ember is "data down, actions up." You might see this concept abbreviated as DDAU in various Ember communities. The main premise of DDAU is that data should flow down through your component hierarchy (passed through and potentially modified by various components), while changes to said data should be propagated back up through that hierarchy via actions.
Many web interfaces let a user effortlessly page through large sets of data. Implementing database queries that fetch these pages is also effortless for the programmer, usually requiring an OFFSET and LIMIT in the case of SQL and a FROM and SIZE in the case of Elasticsearch. Although this method is easy on the user and programmer, pagination queries of this type have a high hidden cost for SQL, Elasticsearch and other database engines.
At Salsify, we encountered this problem when implementing a feature to allow a user to step through records in a large, heavily filtered and sorted set. We had to implement an efficient pagination solution that would work in both our SQL and ES datastores. In this post we’ll look at an interesting technique to make pagination efficient for the database with large datasets. Specifically, we’ll look at implementation in SQL as well as how to translate this method to Elasticsearch.
Have you ever noticed that no one writes "How we name ourRuby variables at Company X" blog posts? No one's making the Hacker News front page with "I combined these two strategies for method naming and suddenly my JavaScript is maintainable!" And yet when it comes to CSS, developers are all about naming strategies. We sing the praises of BEM, SMACSS, OOCSS, SUIT, or whatever other set of capital letters is popular this week. Why is that?
Using vim has been one of the greatest productivity boosters for my development life. I got into vim as a lowly system administrator because it seemed to be the tool of the trade. From there, my knowledge grew and now it is my editor of choice for almost all projects. This post will go through my current setup for Ruby on Rails development. I'd like to give a huge shout out to @tpope who has created a plethora of amazing plugins all worth making a part of your daily workflow.
Here at Salsify, many of our customers regularly import large amounts of tabular product data into the system. This data needs to be sorted prior to being handled by different parts of the import process. Since we are running on Heroku, memory is a scarce resource. Sorting these arbitrarily large tabular data files requires great care. Reading all of the data into memory at once can result in extremely long execution times due to increased pressure on the Ruby garbage collector and can starve other processes on the same system of memory.
We needed a way to sort large files using a predictable amount of memory. Big data technologies like map reduce were overkill for our data scale. Creating an offline-sort gem to do this turned out to be quite the adventure and forced me to dig deeper into how Ruby manages memory, ultimately requiring a specialized heap implementation.