Keywords: stream processing, big data, ETL, scale
Webpages:
https://CRAN.R-project.org/package=AWR,
https://CRAN.R-project.org/package=AWR.KMS,
https://CRAN.R-project.org/package=AWR.Kinesis R is rarely mentioned among the big data tools, although it’s fairly well scalable for most data science problems and ETL tasks. This talk presents an open-source
R package to interact with Amazon Kinesis via the MultiLangDaemon bundled with the Amazon KCL to start multiple
R sessions on a machine or cluster of nodes to process data from theoretically any number of Kinesis shards.
Besides the technical background and a quick introduction on how Kinesis works, this talk will feature some stream processing use-cases at CARD.com, and will also provide an overview and hands-on demos on the related data infrastructure built on the top of Docker, Amazon ECS, ECR, KMS, Redshift and a bunch of third-party APIs – besides the related open-source
R packages, eg
AWR,
AWR.KMS and
AWR.Kinesis, developed at CARD.
References
