• Stateful adaptive streams with...

Treffer: Stateful adaptive streams with approximate computing and elastic scaling

Title:
Stateful adaptive streams with approximate computing and elastic scaling
Authors:
Francisco, João, Coimbra, Miguel E., Neto, Pedro F., Freitag, Fèlix, Veiga, Luis
Contributors:
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors, Universitat Politècnica de Catalunya. CNDS - Xarxes de Computadors i Sistemes Distribuïts
Publisher Information:
Association for Computing Machinery (ACM)
Publication Year:
2023
Collection:
Universitat Politècnica de Catalunya, BarcelonaTech: UPCommons - Global access to UPC knowledge
Subject Terms:
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors::Arquitectures distribuïdes, Electronic data processing -- Distributed processing, Graph theory, Adaptive stream processing, Approximate computation, Stateful functions, Apache flink, Processament distribuït de dades, Grafs, Teoria de
Document Type:
Konferenz conference object
File Description:
10 p.; application/pdf
Language:
English
Relation:
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2019-106774RB-C21/ES/SISTEMAS INFORMATICOS Y DE RED DESCENTRALIZADOS CON RECURSOS DISTRIBUIDOS/; info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PCI2019-111850-2/ES/PROCESAMIENTO DE FLUJO DISTRIBUIDO EN SISTEMAS DE NIEBLA Y BORDE MEDIANTE COMPUTACION TRANSPRECISA/; http://hdl.handle.net/2117/399792
DOI:
10.1145/3555776.3577858
Availability:
http://hdl.handle.net/2117/399792
https://doi.org/10.1145/3555776.3577858
Rights:
Open Access
Accession Number:
edsbas.9E4D1055
Database:
BASE

Weitere Informationen

The model of approximate computing can be used to increase performance or optimize resource usage in stream and graph processing. It can be used to satisfy performance requirements (e.g., throughput, lag) in stream processing by reducing the effort that applications need to process datasets. There are currently multiple stream processing platforms, and most of them do not natively support approximate results. A recent one, Stateful Functions, is an API that uses Flink to enable developers to easily build stream and graph processing applications. It also retains Flink's features like stateful computations, fault-tolerance, scalability, control events and its graph processing library Gelly. Herein we present Approxate, an extension over this platform to support approximate results. It can also support more efficient stream and graph processing by allocating available resources adaptively, driven by user-defined requirements on throughput, lag, and latency. This extension enables flexibility in computational trade-offs such as trading accuracy for performance. The user can choose which metrics should be guaranteed at the cost of others, and/or the accuracy. Approxate incorporates approximate computing (using load shedding) with adaptive accuracy and resource manegement in state-of-the-art stream processing platforms, which are not targeted in other relevant related work. It does not require significant modifications to application code, and minimizes imbalance in data source representation when dropping events. ; This work was supported by national funds through FCT, Fundação para a Ciência e a Tecnologia, under project UIDB/50021/2020. DL 60/2018, de 3-08 – Aquisição necessária para a atividade de I&D do INESC-ID, no âmbito do projeto SmartRetail (02/C-05i01/2022). This work was partially supported by the Spanish Government under research contracts PID2019-106774RB-C21 and PCI2019-111850-2 (DiPET CHIST-ERA). ; Peer Reviewed ; Postprint (author's final draft)