Data processing is a common task in development which can often be solved through processing pipelines. Like the name implies, a pipeline is made up of a series of pipes where the output type of one pipe is the input type for its adjacent pipe. A simple, light weight, and powerful language construct for building pipes are coroutines, which can suspend and resume operation while maintaining state. If you're developing in a language with native coroutine support—like Python—then you can put that scaffolding to work developing the processing pipeline of your dreams. However, if you're not fortunate enough to have access to a native coroutine construct, they still serve as a great source for inspiration when developing your own solution.
Posts Tagged 'java'
While working on a small command line Java application (which will
appear in a later post) I found myself needing to concurrently save lots
and lots of files to a ZIP archive. Prior to the NIO package that
debuted in Java 7 this wasn't something easily done. In this post we'll
use some of the fundamental APIs from the NIO package along with
concurrency utilities to quickly traverse a directory tree and copy file
contents into a ZIP archive. However, to truly appreciate the speed
boost from NIO's asynchronous file channels, we'll first visit and
benchmark a more "vintage" approach leveraging the