Reinforcement Learning for Predictive Analytics in Smart Cities

Journal Title: Informatics - Year 2017, Vol 4, Issue 3


The digitization of our lives cause a shift in the data production as well as in the required data management. Numerous nodes are capable of producing huge volumes of data in our everyday activities. Sensors, personal smart devices as well as the Internet of Things (IoT) paradigm lead to a vast infrastructure that covers all the aspects of activities in modern societies. In the most of the cases, the critical issue for public authorities (usually, local, like municipalities) is the efficient management of data towards the support of novel services. The reason is that analytics provided on top of the collected data could help in the delivery of new applications that will facilitate citizens’ lives. However, the provision of analytics demands intelligent techniques for the underlying data management. The most known technique is the separation of huge volumes of data into a number of parts and their parallel management to limit the required time for the delivery of analytics. Afterwards, analytics requests in the form of queries could be realized and derive the necessary knowledge for supporting intelligent applications. In this paper, we define the concept of a Query Controller (QC) that receives queries for analytics and assigns each of them to a processor placed in front of each data partition. We discuss an intelligent process for query assignments that adopts Machine Learning (ML). We adopt two learning schemes, i.e., Reinforcement Learning (RL) and clustering. We report on the comparison of the two schemes and elaborate on their combination. Our aim is to provide an efficient framework to support the decision making of the QC that should swiftly select the appropriate processor for each query. We provide mathematical formulations for the discussed problem and present simulation results. Through a comprehensive experimental evaluation, we reveal the advantages of the proposed models and describe the outcomes results while comparing them with a deterministic framework.

Authors and Affiliations

Kostas Kolomvatsos and Christos Anagnostopoulos


Related Articles

Acknowledgement to Reviewers of Informatics in 2016

The editors of Informatics would like to express their sincere gratitude to the following reviewers for assessing manuscripts in 2016. We greatly appreciate the contribution of expert reviewers, which is crucial to the...

Selective Wander Join: Fast Progressive Visualizations for Data Joins

Progressive visualization offers a great deal of promise for big data visualization; however, current progressive visualization systems do not allow for continuous interaction. What if users want to see more confident...

digiMe: An Online Portal to Support Connectivity through E-Learning in Medical Education

Connectivity is intrinsic to all aspects of our life today, be it political, economic, technological, scientific, or personal. Higher education is also transcending the previous paradigm of technology enabled content d...

Player–Game Interaction and Cognitive Gameplay: A Taxonomic Framework for the Core Mechanic of Videogames

Cognitive gameplay—the cognitive dimension of a player’s experience—emerges from the interaction between a player and a game. While its design requires careful consideration, cognitive gameplay can be designed only ind...

Web Apps Come of Age for Molecular Sciences

Whereas server-side programs are essential to maintain databases and run data analysis pipelines and simulations, client-side web-based computing tools are also important as they allow users to access, visualize and an...

Download PDF file
  • EP ID EP44101
  • DOI
  • Views 177
  • Downloads 0

How To Cite

Kostas Kolomvatsos and Christos Anagnostopoulos (2017). Reinforcement Learning for Predictive Analytics in Smart Cities. Informatics, 4(3), -.