Data/hora: 20/06/2024 03:25:03

Biblioteca(s): | Embrapa Pecuária Sudeste.
Data corrente: | 10/05/2023
Data da última atualização: | 10/05/2023
Autoria: | VILELA, F. DE A.; TIMES, V. C.; BERNARDI, A. C. de C.; FREITAS, A. DE P.; CIFERRI, R. R.
Afiliação: | FLÁVIO DE ASSIS VILELA, Federal Institute of Goiás; VALÉRIA CESÁRIO TIMES, Federal University of Pernambuco; ALBERTO CARLOS DE CAMPOS BERNARDI, CPPSE; AUGUSTO DE PAULA FREITAS, Federal University of São Carlos; RICARDO RODRIGUES CIFERRI, Federal University of São Carlos.
Título: | A non-intrusive and reactive architecture to support real-time ETL processes in data warehousing environments.
Ano de publicação: | 2023
Fonte/Imprenta: | Heliyon, v. 9, n. 5, e15728, may 2023.
Páginas: | 25 p.
DOI: | https://doi.org/10.1016/j.heliyon.2023.e15728
Idioma: | Inglês
Conteúdo: | Nowadays, organizations are very interested to gather data for strategic decision-making. Data are disposable in operational sources, which are distributed, heterogeneous, and autonomous. These data are gathered through ETL processes, which occur traditionally in a pre-defined time, that is, once a day, once a week, once a month or in a specific period of time. On the other hand, there are special applications for which data needs to be obtained in a faster way and sometimes even immediately after the data are generated in the operation data sources, such as health systems and digital agriculture. Thus, the conventional ETL process and the disposable techniques are incapable of making the operational data delivered in real-time, providing low latency, high availability, and scalability. As our proposal, we present an innovative architecture, named Data Magnet, to cope with real-time ETL processes. The experimental tests performed in the digital agriculture domain using real and synthetic data showed that our proposal was able to deal in real-time with the ETL process. The Data Magnet provided great performance, showing an almost constant elapsed time for growing data volumes. Besides, Data Magnet provided significant performance gains over the traditional trigger technique.
Palavras-Chave: | Data extraction; Data loading; Data warehouse; ETL; Real time.
Categoria do assunto: | X Pesquisa, Tecnologia e Engenharia
URL: | http://ainfo.cnptia.embrapa.br/digital/bitstream/doc/1153634/1/NonIntrusiveReactive.pdf
Registro original: | Embrapa Pecuária Sudeste (CPPSE)
Exemplares: | CPPSE - (26006-1) - UPC - AP - DD - PROCI-2023.00033 - VIL - 2023.00099 - DIS