Across Domains

Unsupervised and Reinforcement Learning "Across Domains"

 

Unsupervised Learning "Across Domains"

We have seen that unsupervised learning algorithms can be applied to a variety of tasks. Previously, you probably discussed with your peers how unsupervised learning can be used for community detection in social networks. In social networks, people are constantly grouped based on their similar preferences, decisions, and tastes, leading to the formation of virtual clusters or communities. Detecting such communities can be useful in many areas – for example, to identify a shared research field in collaboration networks (like ResearchGate), to find a group of like-minded users for marketing and recommendations, or to identify protein interaction networks in biological systems. With unsupervised learning algorithms, it is possible to group similar individuals based on their features – such as age, gender, or preferences – by calculating their similarity.

Unsupervised learning models are used to assist professionals from various fields in performing different tasks. Please choose one of the areas below that is closest to your field and explore different tasks that can be supported by unsupervised learning models:

Medicine

How can we predict diseases with unsupervised learning?
In healthcare, there is growing interest in using unsupervised learning approaches, especially in situations where data labels may be missing – such as with undiscovered or rare diseases. For example, the study by Lu and Uddin (2024) used unsupervised learning algorithms to develop disease prediction models. Specifically, the study examined the effectiveness of seven unsupervised algorithms on 15 datasets, including those on heart failure, diabetes, and breast cancer.


How can we detect anomalies in medical images?
Anomaly detection using unsupervised learning methods plays a crucial role in radiology to identify unusual patterns in medical images that may indicate diseases such as tumors or fractures. Techniques like autoencoders can be trained with normal image data to learn typical features. Deviations can then be detected when analyzing new images. This approach improves diagnostic accuracy and reduces the likelihood of missing critical conditions (Shveotsova et al., 2021).
 


LITERATURE REFERENCES
  1. Lu, Haohui, and Shahadat Uddin. “Unsupervised Machine Learning for Disease Prediction: A Comparative Performance Analysis Using Multiple Datasets.” Health and Technology 14, no. 1 (January 1, 2024): 141–54. https://doi.org/10.1007/s12553-023-00805-8.
  2. N. Shvetsova, B. Bakker, I. Fedulova, H. Schulz and D. V. Dylov, "Anomaly Detection in Medical Imaging With Deep Perceptual Autoencoders," in IEEE Access, vol. 9, pp. 118571-118583, 2021, https://doi.org/10.1109/ACCESS.2021.3107163.

Biology

How can we identify new drug targets using clustering?
In biological research, unsupervised learning is used to analyze large datasets for drug development. By applying clustering algorithms to genomic and proteomic data, researchers can identify potential drug targets that share similar properties or functions. For example, hierarchical clustering can reveal relationships between proteins that may not be detectable using traditional analytical methods (Atas Guvenilir et al., 2023).
 


LITERATURE REFERENCES
  1. Atas Guvenilir, H., & Doğan, T. (2023). How to approach machine learning-based prediction of drug/compound–target interactions. Journal of Cheminformatics, 15(1). https://doi.org/10.1186/s13321-023-00689-w


Economics
How can customers be segmented with unsupervised learning?
Unsupervised learning can be effectively used for customer segmentation by identifying different groups based on shared features and behaviors. For example, in the study by Shen (2021), unsupervised learning algorithms were used to segment customers in a real-world database of an online transaction platform.


How does unsupervised learning support fraud detection?
Unsupervised learning plays a crucial role in fraud detection by identifying anomalies in transaction data that deviate from established normal patterns. Since fraudulent transactions in credit card data increase every year, Rai and Dwivedi (2020) proposed a scheme for credit card fraud detection based on unsupervised learning algorithms.

LITERATURE REFERENCES
  1. Shen, Boyu. “E-Commerce Customer Segmentation via Unsupervised Machine Learning.” In The 2nd International Conference on Computing and Data Science, 1–7. CONF-CDS 2021. New York, NY, USA: Association for Computing Machinery, 2021. https://doi.org/10.1145/3448734.3450775.
  2. Rai, Arun Kumar, and Rajendra Kumar Dwivedi. “Fraud Detection in Credit Card Data Using Unsupervised Machine Learning Based Scheme.” In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), 421–26, 2020. https://doi.org/10.1109/ICESC48915.2020.9155615.

 

Geology
 
 
How can unsupervised learning be applied to investigate data patterns in geophysical datasets?
In the geosciences, the approach of unsupervised learning is used to identify geological units by analyzing complex geophysical datasets. A study by Zhang et al. (2019) applied unsupervised learning algorithms to investigate data patterns in multivariate geophysical datasets obtained from sediment core samples of the seafloor during scientific ocean drilling in the South China Sea. This approach enabled researchers to create exploratory data clusters that corresponded with established lithological units and geological timescales. The results showed that unsupervised learning can effectively reveal relationships within the data and support geological interpretations (Zhang et al., 2019).
 
How can seismic data analysis be supported by unsupervised learning?
Unsupervised learning is increasingly used in the geosciences to analyze seismic data and improve the interpretation of subsurface structures. A study by Celecia et al. (2020) demonstrated that unsupervised learning can analyze large seismic datasets and generate seismic clusters that correctly represent the main seismic facies.

REFERENCES
  1. Zhang, Y., Liu, S., & Zhang, Y. (2019). Unsupervised Learning on Scientific Ocean Drilling Datasets from the South China Sea. Journal of Geomechanics, 14(3), 205-215. https://doi.org/10.1007/s11707-018-0704-1.
  2. Celecia, A., Figueiredo, K., Rodriguez, C., Vellasco, M., Maldonado, E., Silva, M. A., Rodrigues, A., Nascimento, R., & Ourofino, C. (2021). Unsupervised Machine Learning Applied to Seismic Interpretation: Towards an Unsupervised Automated Interpretation Tool. Sensors, 21(19), 6347. https://doi.org/10.3390/s21196347
 

 

Reinforcement Learning "Across Domains"

We have already discussed that a robot can learn tasks through the reinforcement learning approach. A robot learns through trial and error: it receives positive rewards when it successfully performs tasks—such as balancing on two legs or assembling a product—and penalties for mistakes, such as tipping over or dropping an object. Over time, the robot refines its behavior and becomes more efficient, adaptable, and capable of autonomously handling even complex tasks.

Reinforcement learning models are also used to assist professionals from various fields in performing different tasks. Below, choose one of the areas closest to your field and explore different tasks that can be supported by reinforcement learning models:

Medicine
 
How can reinforcement learning support disease diagnosis? 
Reinforcement learning has shown potential in improving breast cancer screenings (Yala et al., 2022). Another prominent example is the application of a reinforcement learning algorithm with expert-assisted rewards for diagnosing skin cancer (Barata et al., 2023).
 
How can reinforcement learning optimize medication dosing in intensive care?
Reinforcement learning has been used to develop AI systems aimed at optimizing medication dosages. For example, reinforcement learning was applied in simulated environments to optimize personalized dosing of propofol, a commonly used sedative in intensive care units, ensuring adequate sedation (Borera et al., 2011). Reinforcement learning has also been used to create individualized chemotherapy plans for cancer patients and develop strategies to optimize chemotherapy dosing (Ebrahimi Zade, Shahabi Haghighi, and Soltani, 2020).




REFERENCES
  1. Yala, Adam, Peter G. Mikhael, Constance Lehman, Gigin Lin, Fredrik Strand, Yung-Liang Wan, Kevin Hughes, et al. “Optimizing Risk-Based Breast Cancer Screening Policies with Reinforcement Learning.” Nature Medicine 28, no. 1 (January 2022): 136–43. https://doi.org/10.1038/s41591-021-01599-w.
  2. Barata, Catarina, Veronica Rotemberg, Noel C. F. Codella, Philipp Tschandl, Christoph Rinner, Bengu Nisa Akay, Zoe Apalla, et al. “A Reinforcement Learning Model for AI-Based Decision Support in Skin Cancer.” Nature Medicine 29, no. 8 (August 2023): 1941–46. https://doi.org/10.1038/s41591-023-02475-5.
  3. Borera, E. C., Moore, B. L., Doufas, A. G. & Pyeatt, L. D. An Adaptive Neural Network Filter for Improved Patient State Estimation in Closed-Loop Anesthesia Control. in 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence 41-46 (2011). http://doi.org/10.1109/ICTAI.2011.15
  4. Ebrahimi Zade, A., Shahabi Haghighi, S. & Soltani, M. Reinforcement learning for optimal scheduling of glioblastoma treatment with temozolomide. Computer Methods Prog. Biomedicine 193, 105443 (2020). https://doi.org/10.1016/j.cmpb.2020.105443

Biology
 
How can reinforcement learning optimize experimental design in biology? 
Reinforcement learning (RL) can significantly improve experimental design in biological research by enabling the identification of optimal experimental conditions that maximize information from each experiment. A study by Treloar et al. (2022) demonstrated the use of deep reinforcement learning for optimal design of biological experiments, particularly for deriving parameters of bacterial growth in a simulated chemostat. The researchers formulated the problem as an RL task, where the agent learns to select experiments that provide the most informative data regarding model parameters. By incorporating uncertainty in model parameterization, the RL approach outperformed traditional optimization methods and showed its potential to optimize experimental workflows and enhance the efficiency of biological research.
 
How can reinforcement learning facilitate protein engineering?
Reinforcement learning has proven to be a powerful tool for advancing protein engineering by enabling the generation of new protein architectures tailored to specific functions. A notable example is the research by Lutz et al. (2023), where a reinforcement learning approach was used to optimize protein structures based on desired properties such as stability and binding affinity. The RL model was trained on an extensive library of protein sequences and iteratively refined its designs through trial-and-error to achieve predefined goals. This innovative approach not only led to the successful creation of proteins generating useful antibodies but also demonstrated RL’s potential to revolutionize protein engineering, allowing scientists to explore a broader range of molecular configurations than traditional methods.




REFERENCES
  1. Treloar, N. J., Braniff, N., Ingalls, B., & Barnes, C. P. (2022). Deep reinforcement learning for optimal experimental design in biology. PLoS Computational Biology, 18(11), e1010695. http://doi.org/10.1371/journal.pcbi.1010695.
  2. Lutz, I. D., Wang, S., & Baker, D. (2023). Top-down design of protein architectures with reinforcement learning. Science, 380(6639), 1234-1239. http://doi.org/10.1126/science.adf6591.

Chemistry
 
How can reinforcement learning optimize chemical reactions? 
Reinforcement learning (RL) has been successfully applied to optimize chemical reactions by enabling adaptive decision-making based on experimental results. A notable study by Zhou et al. (2017) introduced a Deep Reaction Optimizer that used deep reinforcement learning to iteratively improve the conditions of chemical reactions. The model recorded the results of each reaction and adjusted the experimental parameters to increase yield. By implementing an efficient exploration strategy, the RL algorithm reduced the number of experimental steps required to achieve optimal conditions by 71% compared to traditional optimization methods. This approach not only accelerated the optimization process and achieved optimal conditions in just 30 minutes, but also demonstrated the model’s ability to learn from both similar and different reactions, highlighting its versatility in various chemical contexts.
 


REFERENCES
  1. Zhou, Z., Li, X., & Zare, R. N. (2017). Optimizing Chemical Reactions with Deep Reinforcement Learning. ACS Central Science, 3(12), 1337-1344. http://doi.org/10.1021/acscentsci.7b00492.

Economics
 
How can reinforcement learning optimize financial trading strategies?  
Reinforcement learning (RL) can be used to optimize trading strategies in finance. For example, in Huang et al., 2024, reinforcement learning was applied using multiple intelligent agents to learn from and interact with the environment, improving decision-making processes and overall trading performance.
 


REFERENCES
  1. Huang, Y., Zhou, C., Cui, K., & Lu, X. (2024). A multi-agent Reinforcement Learning Framework for optimizing financial trading strategies based on TimesNet. Expert Systems with Applications, 237, 121502. https://doi.org/10.1016/j.eswa.2023.121502

Geology
 
How can reinforcement learning improve mineral exploration in geoscience? 
Reinforcement learning can also support mineral exploration by optimizing the decision-making process for identifying potential mining sites. A study by Shi et al. (2023) applied reinforcement learning to create a suitable environment for the agent to learn mineralization information. The results can support further gold prospecting.
 


REFERENCES
  1. Shi, Z., Zuo, R. & Zhou, B. Deep Reinforcement Learning for Mineral Prospectivity Mapping. Math Geosci 55, 773–797 (2023). https://doi.org/10.1007/s11004-023-10059-9

Journalism
 
How can reinforcement learning improve content recommendation in journalism? 
Reinforcement learning can enhance recommendation systems in journalism by dynamically adapting to user preferences and behavior over time. Bangari et al., 2021, provided a review highlighting the potential of reinforcement learning for personalized news recommendations.
 
How can reinforcement learning help in fake news detection?
Reinforcement learning can also help improve systems for detecting fake news by leveraging user-generated feedback to refine model accuracy. Wang et al. (2020) proposed a weakly supervised fake news detection framework that integrates RL techniques to enhance the quality of training data. The framework uses user reports as weak supervision to label unlabeled news articles, and a reinforcement-based selector filters out low-quality samples from the dataset. By applying RL, the model learns to prioritize high-quality inputs that improve its prediction performance over time. This iterative learning process allows the system to adapt to the rapidly changing online news landscape and ensures it remains effective in identifying misinformation even with new trends (Wang et al., 2020).



REFERENCES
  1. Bangari, Sindhuja, Shantharam Nayak, Ladly Patel, and K T Rashmi. “A Review on Reinforcement Learning Based News Recommendation Systems and Its Challenges.” In 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 260–65, 2021. https://doi.org/10.1109/ICAIS50930.2021.9395812.
  2. Wang, Yaqing & Yang, Weifeng & Ma, Fenglong & Xu, Jin & Zhong, Bin & Deng, Qiang & Gao, Jing. (2020). Weak Supervision for Fake News Detection via Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 34. 516-523. http://doi.org/10.1609/aaai.v34i01.5389. 

!     Reflect on other problems in your field of study where unsupervised or reinforcement learning models can be applied.