User-Defined Algorithms on Big Data with OmniSci - Seguridad Mania.com - España y América Latina

OmniSci is designed for big data but for custom computations the data needs to be retrieved to a client machine. This puts a ceiling on data sizes based on both network speed and local hardware resources. What if we could send the algorithm to the data instead? This talk presents the Remote Backend Compiler (RBC), a package that affords OmniSci database capabilities to execute functions written in Python inside SQL queries. Our approach uses Numba, a high performance Python compiler, to generate fast low level machine code of user defined functions. We will learn more about how the RBC connects open source technologies, like NumPy and Numba, to the OmniSci database. Lastly, we conclude with a demonstration using an applied example of the Black-Scholes financial model. This will highlight how to leverage the newly added User-Defined Functions and User-Defined Table Functions capabilities with OmniSci.