Raku
Multi-language Data Wrangling and Acquisition Conversational Agents
Using Raku in data acquisition and wrangling
<p>In this presentation we discuss the Conversational Agent (CA) designs for two closely related problem areas:</p>
<ul>
<li><p>Data Acquisition Workflows (DAWs)</p></li>
<li><p>Data Transformation Workflows (DTWs)</p></li>
</ul>
<p>The CA perspective is taken mostly for exposition and didactic purposes. Nevertheless, we emphasise the practical applicability of the underlying designs and implementations.</p>
<p>Although, operationally data acquisitions are prerequisite for data wrangling we discuss data wrangling first -- the corresponding DTWs designs and implementations are more mature and the related materials are more universal, applicable to multiple programming languages.</p>
Multi-language Data Wrangling and Acquisition Conversational Agents
Anton Antonov
FOSDEM 2022
Abstract
In this presentation we discuss the Conversational Agent (CA) designs for two closely related problem areas:
Data Acquisition Workflows (DAWs)
Data Transformation Workflows (DTWs)
The CA perspective is taken mostly for exposition and didactic purposes. Nevertheless, we emphasise the practical applicability of the underlying designs and implementations.
Although, operationally data acquisitions are prerequisite data wrangling we discuss data wrangling first -- the corresponding DTWs designs and implementations are more mature and the related materials are more universal, applicable to multiple programming languages.
Outline
Data Wrangling
In the first part of the presentation we show and compare data wrangling examples in different programming languages using different packages.
Here is a list of the programming languages and packages we consider:
Julia-DataFrames
Python-pandas
R
R-tidyverse
WL
We look into the common data wrangling workflows and how we can design a conversational agent that translates natural language commands into data wrangling code for Julia, Python, R, SQL, WL.
WL's external evaluator features are heavily utilized.
Data Acquisition Workflows
In the second part of the presentation we discuss the following facets of a data acquisition system:
Conversational Agent based on a Finite State Machine
Gathering and utilizing metadata taxonomies
The making of datasets recommender systems and search engines
In/for both R and WL
Making (ingredient) variables queries
Introspection queries
Random data generation specifications
Data obfuscation specifications
Extensions to ML models acquisition workflows
Additional information
| Type | devroom |
|---|
More sessions
| 2/5/22 |
<p>Welcome to the Raku devroom at FOSDEM 2022!</p>
|
| 2/5/22 |
<p>One feature that sets raku apart from other programming languages is the built-in Grammar parser syntax. This provides a concise, clean and extensible tool for parsing text and triggering Actions according to the content. Navigation aids such as buoys and markers sport flashing lights with characteristics such as color, duration, phase, occulting, speed, height, visibility and so on are represented on navigation charts by way of a short code e.g. <code>Fl(4)15s37m28M</code>.</p> <p>This talk ...
|
| 2/5/22 |
<p>Writing Free Software is very different from writing proprietary software for a huge software company like Google or Facebook – the team size, incentives, and likelihood of turnover are all completely different – and, as a result, the ideal languages are very different as well. Or at least, this was the thesis for my 2021 FOSDEM talk "<a href="https://archive.fosdem.org/2021/schedule/event/programming_lang_for_free_software/">Imagining the Ideal Language for Writing Free ...
|
| 2/5/22 |
<p>Rakulang is great not only as a scripting language, but also at the conceptual level, allowing you to map problem-domain concepts to data structures easily. This comes handy when you use a specific kind of learning procedure that needs to track general class progress as well as individual attainment levels in a very precise way. In this context, we have created a Raku mini-library and a series of scripts that give us information on how a whole class of Cloud Computing students is progressing, ...
|
| 2/5/22 |
<p>Authentication is the «must-have» feature of every web application. Traditional centralized auth feature has the major disadvantage: if the authenticator is down, it affects on to all related services. In this talk I would like to discuss auth on Ethereum blockchain, go through its major benefits and introduce sample application with the decentralized authentication implemented in Raku.</p>
|
| 2/5/22 |
<p>The Raku ecosystem is growing at an impressive rate. The Raku Programming Language allows developers to specify the exact version of a module that is needed in production. This is a very good thing, and makes sysadmins all over the world pretty happy, as they know they can install new versions of modules without fear of breaking production code.</p> <p>However, developers can also <em>remove</em> distributions from the ecosystem. This presentation describes efforts to make sure that this will ...
|
| 2/5/22 |
<p>Some languages offer a very simple syntax for testing via assertion statements, for example with an <code>assert</code> function that creates a simple test that passes if given a true value and fails for anything else. This style of testing is admirably concise, but can lead to cryptic test messages that make it very difficult to determine what a test was attempting to check without opening up the file containing the failing test. At the other extreme, it's possible to write very expressive ...
|
