Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review (2024)

1 Introduction

Language represents a rigorously structured communicative system characterized by its grammar and vocabulary. It serves as the principal medium through which humans articulate and convey meaning. This conception of language as a structured communicative tool is pivotal in the realm of computational linguistics, particularly in the development and evaluation of natural language processing (NLP) algorithms. A seminal aspect in this field is the Turing Test, proposed by Alan Turing in 1950 [1], which evaluates a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. In this context, the Turing Test primarily assesses the machine’s capability to perform tasks involving language comprehension and generation, reflecting the intricate role of linguistic structure in the artificial replication of human-like communication.

Language model (LM) is a fundamental element employed in a multitude of NLP tasks, such as text generation, machine translation, and speech recognition [2, 3]. These models are intricately designed to comprehend, generate, and manipulate human language. The training of language models involves large-scale corpora, enabling them to learn universal language representations. This training process is critical for the models to capture the semantics of words in varying contexts [4, 5, 6]. Notably, the fidelity of these representations is frequently contingent on the word frequency within the training corpus. Such dependency underscores the importance of a comprehensive and diverse corpus in training LMs, as it directly impacts their ability to reflect and understand the nuances of natural language accurately. The intricacy of language models and their reliance on corpus characteristics are vital considerations in advancing NLP technologies, which underscores the significance of human-like language comprehension and production in artificial intelligence systems.

The forefront of advancements in language model technology has been marked by the emergence of Large Language Models (LLMs). This evolution signifies a paradigm shift in the field of NLP and extends its impact to broader applications. LLMs leverage deep learning methodologies [7], utilizing extensive datasets to perform complex tasks such as understanding, summarizing, generating, and predicting novel content. These models operate by processing an input text and iteratively predicting subsequent tokens or words. A distinguishing feature of LLMs is their vast parameter space, encompassing tens to hundreds of billion parameters, in stark contrast to their predecessors [4, 3]. In addition, they are trained on significantly larger datasets, ranging from several gigabytes to terabytes in size. This exponential increase in both computational capacity and training data volume has not only enhanced the performance of LLMs in conventional NLP tasks but also has expanded their applicability in areas such as contextual analysis and sentiment detection.The advancements in LLMs reflect the ongoing pursuit of achieving and surpassing human-level proficiency in language understanding and generation.

Forecasting and anomaly detection represent pivotal components in the realm of data science, delivering essential insights across a multitude of domains ranging from network security to financial markets [8, 9, 10, 11, 12, 13, 14]. These techniques are integral in projecting forthcoming trends and pinpointing atypical patterns that diverge from normative expectations. Such capabilities are proactive in fostering preemptive strategies in a wide array of applications.

Forecasting uses historical data to make informed predictions about future occurrences or trends. It involves making assumptions about the situation being analyzed, selecting an appropriate data set, analyzing the data, and determining the forecast. Forecasting serves as a cornerstone for strategic planning and decision-making in diverse sectors, ranging from economics and finance to healthcare and environmental management, that empowers organizations and policymakers to anticipate changes, manage risks, and allocate resources efficiently [15, 16, 17]. In the financial sector, for instance, accurate forecasting is essential for investment strategies, risk management, and market analysis [18, 19]. It enables investors and financial analysts to predict market trends, assess the viability of investments, and mitigate potential risks. Similarly, in supply chain management, forecasting plays a pivotal role in inventory control, demand planning, and logistics optimization, thus ensuring operational efficiency and cost-effectiveness [20]. Moreover, in the realm of public policy and healthcare, forecasting is critical for preparing for future demands, whether it be anticipating economic shifts, public health needs, or environmental changes [21]. Accurate predictions can guide policy formulation and resource allocation, thereby enhancing the effectiveness of public services and interventions.

Anomaly detection, also known as outlier detection, is an analytical process aimed at identifying data points, entities, or occurrences that exhibit significant deviations from the typical patterns or norms [22, 23].This methodology plays a critical role in automated surveillance systems, particularly in identifying potentially detrimental outliers, thereby safeguarding data integrity and security [24].Its application is especially crucial in sectors such as finance [25], retail [26], and cybersecurity [27, 28].In the financial industry, anomaly detection is instrumental in fraud detection and anti-money laundering efforts.It enables financial institutions to quickly identify unusual transaction patterns that may indicate fraudulent activity, thereby protecting both the institution and its customers from financial loss [25, 29].Similarly, in the retail sector, anomaly detection can highlight unusual purchasing patterns or inventory issues, assisting in loss prevention and optimizing supply chain management [26]. The field of cybersecurity significantly benefits from anomaly detection. It is used to identify unusual network traffic, access patterns, or system behavior that could signify a security breach or cyberattack [27, 30]. By detecting these anomalies early, organizations can rapidly respond to potential threats, mitigating the risk of data breaches and cyberattacks [31, 32].

Forecasting and anomaly detection are analytical processes inherently well-suited for time series or timestamped data due to the temporal nature of the information they seek to understand and leverage. Time series data, by definition, is a sequence of data points collected or recorded at time intervals, which often exhibits trends, seasonal variations, and cycles that forecasting techniques aim to capture and extrapolate into the future [24, 33]. Timestamped data is particularly conducive to anomaly detection because it allows for the recognition of deviations from established temporal patterns. For instance, in cybersecurity, anomaly detection systems can identify unusual access patterns that may indicate a security breach [10]. In industrial settings, it might flag an unexpected drop or spike in sensor readings, potentially preventing equipment failure. Figure 1 depicts an overview of leveraging a large language model for forecasting and anomaly detection tasks. The input data is often time series or timestamped data, encompassing a variety of formats such as text logs, numerical data, structured data, graphical input, and speech recordings. Current widely used LLMs such as BERT [2], GPT [34], LLaMA2 [35], and Mixtral [36] are transformer-based, which includes mechanisms such as input and output embeddings, multi-head attention, and feed-forward neural networks. Forecasting tasks involve predicting future data points based on learned patterns, while anomaly detection identifies outliers or unexpected events in the data stream.

In this study, we embark on a comprehensive exploration of the integration and potential of LLMs in the realms of forecasting and anomaly detection, areas traditionally dominated by quantitative data analysis. The rapid evolution of LLMs in NLP presents an unprecedented opportunity to augment and possibly revolutionize these domains. This paper aims to bridge the gap between the advanced linguistic processing capabilities of LLMs and the predictive analytics involved in forecasting and detecting outliers. We delve into how the qualitative insights gleaned from LLMs can complement traditional quantitative approaches, thereby enriching the analytical depth and accuracy in various sectors, including finance, cybersecurity, and healthcare. Additionally, this survey addresses the challenges, ethical considerations, and future research trajectories at the intersection of LLMs with these critical data science applications. Our objective is to provide a holistic view that not only elucidates the current state of LLM applications in these fields but also stimulates interdisciplinary dialogue and research, navigating the complexities of modern data environments and paving the way for innovative solutions in predictive analytics.

Contributions. To recapitulate, we highlight the following contributions:

2 Methodology

In the rapidly evolving domain of artificial intelligence (AI), LLMs have emerged as copilot tools in various applications, notably in forecasting and anomaly detection [10, 23]. However, despite their growing prominence, a substantial knowledge gap exists regarding their comprehensive capabilities and limitations in these contexts. This review is motivated by the necessity to consolidate and critically analyze the extant research concerning LLMs in these specific applications. In light of the rapid progress in model architectures and their diverse applications, this review aims to amalgamate knowledge on existing methodologies, performance evaluation metrics, and practical implementations while also identifying the prevailing challenges and limitations. This effort is imperative for both academic researchers and industry practitioners who aim to utilize these models effectively and serves as a foundational reference for future research and development in this field. By systematically examining and integrating diverse findings from recent studies, this review aims to offer a structured and comprehensive understanding of the current state-of-the-art, thereby guiding informed decision-making and strategic advancements in the application of LLMs.

In our study, we adopted the SLR methodology as outlined by Barbara Kitchenham [37, 38]. This method is a comprehensive, rules-driven approach to finding and analyzing prior knowledge on a particular topic that involves a rigorous and transparent methodology to identify, evaluate, and interpret all available research relevant to a particular research question, topic area, or phenomenon of interest. It is designed to provide an exhaustive overview of the current state of research by integrating findings from various studies [39]. The SLR methodology is widely recognized and extensively applied in numerous academic surveys [40, 41, 42, 43, 44]. The research questions (RQs) that guide our SLR process are presented below:

RQ1 calls for a detailed exploration of the strategies, techniques, andmodels used in applying LLMs across various sectors for predictive purposes.RQ2 seeks to evaluate and compare the performance of LLMs againstconventional techniques in identifying irregularities or unexpected patterns indata.RQ3 necessitates a comprehensive exploration of the obstacles and constraints faced when employing these advanced models in specific predictive and analytical tasks.

After delineating the research questions, we strategically integrate various search engines and databases to identify pertinent studies, as outlined in Table 1. In order to find the most cutting-edge papers, we added OpenReview to the search for forthcoming papers that provide significant insight or data.

After retrieving studies through our established search strategy, we conducted a relevance assessment based on the inclusion and exclusion criteria outlined in Table 2. This process enabled the selection of primary studies offering direct evidence pertinent to the research questions.

The selection process involved filtering papers published within a defined recent time frame to guarantee that the review accurately represented the current landscape of technology and research. This approach was adopted as studies outside the specified time frame may not accurately reflect current technologies and methodologies. Our selection criteria prioritized peer-reviewed articles, conference proceedings, and academic journals to maintain research credibility and rigor. In cases of multiple publications reporting identical research or data (e.g., a paper has an updated extended version), the most recent publication was chosen to eliminate redundancy.

Table 3 provides a comprehensive overview of recent research studies focusing on the application of LLMs for forecasting and anomaly detection tasks. It systematically categorizes each piece of research according to the type of LLMs employed, the specific tasks addressed (forecasting, anomaly detection, or both), the methodological approach (e.g., zero-shot, few-shot, fine-tuning, foundation model, prompt-based), the datasets utilized in the studies, and the performance metrics used to evaluate the models’ effectiveness.

The subsequent sections of this review delve into the detailed analysis of the methodologies, challenges, datasets, and performance metrics employed in LLM-based forecasting and anomaly detection. We also discuss the specific applications of LLMs in these domains, highlighting the current state of research, inherent challenges, and prospective future directions.

3 Overview

The expansive domain of LLMs has ushered in unprecedented advancements in natural language processing, significantly impacting various tasks including forecasting and anomaly detection. This section provides a comprehensive overview of the current state and evolution of LLMs, delineating their foundational structures, development trajectories, and the pivotal role they play in transforming data analysis and predictive modeling. Beginning with a background on LLMs, we trace the evolution of language models from their nascent stages to the sophisticated pre-trained foundation models that serve as the backbone for contemporary applications. We then categorize tasks where LLMs have shown remarkable efficacy, specifically focusing on forecasting and anomaly detection, to illustrate the breadth of their applicability. Further exploration is dedicated to the diverse approaches employed to harness the power of LLMs, including prompt-based techniques, fine-tuning mechanisms, the utilization of zero-shot, one-shot, and few-shot learning, reprogramming strategies, and hybrid methods that combine multiple approaches for enhanced performance. This section aims to equip readers with a thorough understanding of the intricate landscape of LLMs, setting the stage for deeper exploration of their capabilities and applications in the subsequent sections.

4 Challenges

In the realm of forecasting and anomaly detection, the deployment of LLMs represents a paradigm shift towards leveraging vast amounts of data for predictive insights. However, this approach is fraught with significant challenges that stem from the inherent properties of time series data, the lack of labeled instances, the prevalence of missing values, and the complexity of processing noisy and unstructured text data. These obstacles necessitate a sophisticated understanding and innovative methodologies to harness the full potential of LLMs in these applications.

The intricate nature of time series data, characterized by complex seasonality and patterns, demands models capable of capturing and forecasting dynamic temporal behaviors. This complexity is compounded by the multifaceted influences affecting time series, including but not limited to economic indicators, weather conditions, and social events, which introduce additional layers of difficulty in modeling efforts. Moreover, the scarcity of labeled data, especially in the context of anomaly detection, poses a significant hurdle. The effectiveness of LLMs in such scenarios is contingent upon developing and applying advanced strategies that can leverage limited annotations to discern patterns indicative of anomalies. Another pervasive issue in time series analysis is the occurrence of missing data, a consequence of various disruptions in data collection and transmission processes.Different from the computer vision models that can be trained from a small amount of data [168, 169], LLMs require huge natural language corpora for training.Addressing this challenge requires robust imputation methods that can seamlessly integrate with LLMs to ensure the integrity and continuity of the data being analyzed. In order to get reproducible and reusable datasets for analytics, the cORe [170] platform can be exploited.Furthermore, the analysis of unstructured text data introduces additional complexity, as such data often contain high noise and irrelevant information. Effective preprocessing and feature extraction methods are imperative to distill valuable insights from unstructured text, necessitating a nuanced approach to understanding and extracting pertinent information.

These challenges underscore the necessity for innovative solutions that adapt to the complexities of time series data and unstructured text, ensuring that LLMs can be effectively applied to forecasting and anomaly detection tasks. The development of such solutions remains an active area of research, with the potential to significantly advance predictive analytics capabilities.

5 Datasets

In the realm of forecasting and anomaly detection research, the availability of high-quality datasets is a critical factor for advancement. These datasets facilitate rapid development and fine-tuning of effective detection algorithms while also setting benchmarks for evaluating methodological performance. However, the acquisition of such datasets often entails significant financial, material, and workforce investments. The field is currently experiencing early development stages, characterized by challenges such as limited data quantity, complex sample characteristics, and missing labels, both essential for developing effective approaches. This section highlights prominent datasets utilized in LLM for forecasting and anomaly detection, which have been contributed by recent studies. An assessment of these datasets is conducted, pinpointing prevailing limitations and challenges in dataset generation, with the objective of guiding the creation of future datasets in this domain.

6 Evaluation Metrics

Evaluation metrics are indispensable tools for evaluating and comparing models in machine learning and statistical analysis, especially in domains such as forecasting and anomaly detection. In these areas, the ability of a model to predict future values based on historical data or identify irregular patterns that deviate from the norm is critical. Metrics in these contexts serve as quantitative indicators of a model’s performance, offering insights into its predictive accuracy, reliability, and robustness under various conditions.

7 Forecasting with Large Language Models

In the domain of artificial intelligence, LLMs have emerged as pivotal instruments for advancing forecasting methodologies across a myriad of fields. This section delves into the transformative role these models play in predicting future events and trends with unprecedented accuracy. This section is meticulously structured to cover the versatile applications of LLMs in forecasting, starting with time series forecasting, a fundamental approach that is further delineated into short-term and long-term forecasting. Each of these subcategories showcases the specific challenges and solutions that LLMs address, highlighting their flexibility and efficiency. Moving beyond traditional time series analysis, the discussion extends to traffic flow forecasting, illustrating how LLMs enhance urban mobility and reduce congestion through predictive analytics. Furthermore, the section explores the profound impact of LLMs in healthcare clinical prediction, where they offer groundbreaking insights into patient outcomes, disease progression, and treatment efficacy. Through this comprehensive examination, we aim to underscore the significant advancements LLMs bring to forecasting practices, fostering a deeper understanding of their capabilities and potential for innovation in various sectors.

8 Anomaly Detection using Large Language Models

The advent of LLMs has significantly broadened the horizons of anomaly detection, offering sophisticated solutions to identify irregularities across diverse datasets and domains. This section embarks on a comprehensive examination of how LLMs are being utilized to pinpoint deviations that could signify errors, fraud, system failures, or cyber threats. This exploration begins with time series anomaly detection, where LLMs analyze sequential data to detect unusual patterns, benefiting industries reliant on continuous monitoring, such as finance, manufacturing, and energy. Moving forward, the discussion transitions to anomaly log analysis, highlighting the capacity of LLMs to sift through vast quantities of log data to identify and classify anomalies, thereby enhancing IT security and operational efficiency. The section on microservice anomaly detection showcases the application of LLMs in the increasingly complex domain of cloud computing and distributed systems, where they play a crucial role in maintaining system health and security by detecting anomalies at the microservice level. This detailed exploration aims to illuminate the cutting-edge methodologies and impactful applications of LLMs in anomaly detection, underscoring their critical role in safeguarding and optimizing modern digital infrastructures.

9 Threats

In the exploration of leveraging LLMs for forecasting and anomaly detection, several significant challenges and deficiencies have become apparent, shaping the landscape of current methodologies and their practical applications. This section delves into the core threats that hinder the effectiveness and reliability of LLMs in these domains. Firstly, the dependence on extensive historical datasets raises concerns about data availability, quality, and the potential for model bias. The issue of generalizability is also critical, as models often struggle to apply learned patterns across diverse contexts or when encountering novel scenarios. Furthermore, the phenomena of hallucination and robustness underscore the models’ tendencies to generate misleading or inaccurate outputs under certain conditions, questioning their reliability. The knowledge boundary of LLMs, defined by the scope of their training data, presents another fundamental challenge, limiting their ability to generate insights beyond their informational horizon. Lastly, computational efficiency remains a daunting obstacle, as the resource-intensive nature of these models can restrict their accessibility and scalability. Addressing these threats is paramount for advancing the utility of LLMs in forecasting and anomaly detection, necessitating a multifaceted approach to enhance their performance, reliability, and applicability in real-world settings.

10 Future Directions and Trends

As the field of LLMs continues to evolve, its application in forecasting and anomaly detection is poised for transformative advancements. The convergence of technological innovation, research breakthroughs, and interdisciplinary collaboration heralds a future where LLMs can offer unprecedented accuracy, adaptability, and insight. This section outlines key future directions and trends that are expected to shape the utilization of LLMs in these domains.

• Integration of Multimodal Data Sources

The future of LLMs in forecasting and anomaly detection is likely to see a significant shift towards the integration of multimodal data sources. By combining textual data with visual, auditory, and sensor-based information, LLMs can develop a more holistic understanding of complex phenomena. This multimodal approach could enhance the models’ ability to detect nuanced anomalies and forecast events with greater precision, leveraging the complementary strengths of diverse data types, and benefit data automated validation process [206].

• Advancements in Transfer and Meta-Learning

Transfer and meta-learning represent promising avenues for making LLMs more adaptable and efficient. Future developments in these areas could enable models to swiftly adjust to new domains or tasks with minimal additional training. Such capabilities would be invaluable in rapidly changing environments or in applications where data scarcity poses a challenge [207, 208]. By improving the versatility of LLMs, these techniques can expand their applicability across a wider range of forecasting and anomaly detection scenarios.

• Focus on Explainability and Trustworthiness

As LLMs assume a more prominent role in decision-making processes, the demand for explainability and trustworthiness will intensify. Future research is expected to prioritize the development of models that not only perform with high accuracy but also provide transparent and interpretable explanations for their outputs. Enhancing the explainability of LLMs can build trust among users [209], facilitate the identification of biases, and ensure the ethical application of these technologies.

• Medical Analysis

Recent research in medical analysis has achieved notable advancements in image segmentation, classification, and trend prediction through end-to-end applications of deep learning and machine learning techniques [210, 211]. Medical imaging data, including Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Optical Coherence Tomography (OCT) [212], often consist of multilayer scan results, with pathological changes potentially distorting these results. Accurately segmenting or classifying such lesions necessitates extensive training on a large corpus of manually annotated medical images, enabling the model to learn pathological changes end-to-end. Interestingly, the task of identifying pathologies in medical images bears a strong resemblance to abnormality detection utilizing LLMs. This parallel raises a compelling research challenge: how can prior knowledge embedded in LLMs be harnessed to enhance learning efficiency, particularly in the context of limited labeled medical imagery?

Concurrently, advancements in medical imaging technology have significantly enhanced imaging quality and increased the volume of data available [213, 214, 215]. Despite these advances, efforts to integrate medical data with LLMs have primarily focused on individual images [216, 217]. The task of employing LLMs to process and analyze large-scale medical image volumes, which often include multilayer information, continues to pose a significant challenge. Future research directions could explore the development of LLMs specifically tailored to navigate and interpret the complexities introduced by these technological advancements in medical imaging.

As LLMs holds the potential to revolutionize doctor-patient interactions, paving the way for more efficient and comprehensive communication channels. With the assistance of LLMs, healthcare professionals can anticipate improved patient education, enhanced clarity in medical discussions, and streamlined dissemination of complex medical information.

• Real-time Processing and Edge Computing

The ability to process data in real-time and deploy LLMs closer to data sources, such as through edge computing, is set to become a crucial trend. This shift towards real-time analysis and decentralized processing can significantly reduce latency, increase the timeliness of insights, and enable the deployment of LLMs in environments where immediate responses are critical. It also opens the door to new applications in sectors such as finance, healthcare, and manufacturing, where quick decision-making is paramount [218]. As a concrete example, in the realm of indoor positioning, LLMs have the potential to discern patterns in edge data, such as trends in WiFi signals, thereby augmenting the precision of existing WiFi signal-based indoor positioning systems [219]. This capability demonstrates the extensive applicability of LLMs in leveraging real-time edge data for improved accuracy in critical applications.

• Sustainable and Energy-efficient Modeling

As computational demands continue to grow, the sustainability and energy efficiency of LLMs will become a pressing concern. Future trends are likely to include the pursuit of more environmentally friendly models through optimized algorithms, energy-efficient hardware, and practices that minimize the carbon footprint of training and deploying LLMs. This focus on sustainability is essential for ensuring that the benefits of LLMs can be realized without exacerbating environmental impacts.

In conclusion, the trajectory of LLMs in forecasting and anomaly detection is marked by exciting opportunities and challenges. By embracing these future directions and trends, the field can unlock new potentials, addressing pressing issues while paving the way for innovative applications that leverage the full capabilities of LLMs in understanding and predicting complex systems.

• Computer vision

LLMs have made significant strides in aligning with images and performing tasks such as classification and semantic segmentation [220, 221, 222]. However, within the field of computer vision, several low-level visual processing tasks persist, which pose challenges in establishing direct relationships with semantics, thus hindering the direct application of LLMs to these tasks.

These low-level visual tasks encompass various aspects, including denoising, defect detection, multi-view reconstruction, and more [223, 224]. Despite their crucial role in image processing and analysis, these tasks often involve intricate visual patterns and features that are not easily discernible through high-level semantic representations alone. As a result, incorporating LLMs into these tasks remains challenging due to the inherent disparity between low-level visual processing and semantic understanding.

• Collaboration Across Disciplines

The future development and application of LLMs in forecasting and anomaly detection will benefit greatly from increased collaboration across different fields, including multi-core system [225], statistics, machinery [226, 227], robotics [228], other domain-specific areas [229, 230, 231, 232, 233, 234, 235], and ethics [236, 237, 238]. Such interdisciplinary efforts can enrich the models with diverse perspectives and expertise, leading to more robust, innovative, and ethically sound solutions.

11 Related Surveys and Reviews

With the rapid advancement of LLMs, considerable comprehensive reviews have appeared, offering in-depth analyses of different facets of this technology. Zhao et al. [239] provide an extensive overview of LLMs, detailing their background, fundamental discoveries, and core technologies, summarizing a broad spectrum of existing research. Conversely, Yang et al. [240] concentrate on the application spectrum of LLMs across various downstream tasks, highlighting the deployment challenges that accompany their use. Chang et al. [241] focus on the evaluation methodologies for LLMs, exploring the criteria, contexts, and methodologies for assessing their performance in downstream applications and societal impacts. Chang and Bergen [242] delve into the abilities and constraints of LLMs across varied downstream tasks. Huang et al. [243] review the progress in enhancing and assessing the reasoning capabilities of LLMs. These studies collectively address multiple aspects of LLMs, like training, evaluation, and application to different domains. However, before this paper, the burgeoning and promising domain of LLM-based Agents had not been focused specifically. This work compiles over 40 latest relevant works on LLM-based forecaster and anomaly detectors, encapsulating their development, applications, and evaluation processes.

12 Conclusion

This systematic literature review has explored the burgeoning field of LLMs in the context of forecasting and anomaly detection, offering a comprehensive overview of current methodologies, challenges, and future directions. As we have seen, LLMs hold immense potential for transforming these domains, providing sophisticated tools capable of parsing vast datasets to predict future events and identify deviations from norms with remarkable accuracy. However, the journey is fraught with challenges, including the dependence on extensive historical datasets, issues of generalizability, the occurrence of hallucinations, knowledge boundaries, and the need for computational efficiency.

Despite these obstacles, the path forward is illuminated by promising solutions and innovations. The integration of multimodal data sources, advancements in transfer and meta-learning, a focus on explainability and trustworthiness, the push towards real-time processing and edge computing, interdisciplinary collaboration, and a commitment to sustainable modeling practices all represent key trends that will shape the future of LLMs in forecasting and anomaly detection.

The review underscores the importance of continued research and development in this area, highlighting the need for models that are not only powerful and accurate but also transparent, adaptable, and accessible. As technology advances, so too must our approaches to ethical considerations, ensuring that the deployment of LLMs contributes positively to society and does not exacerbate existing inequalities or environmental issues.

In conclusion, the potential of LLMs to revolutionize forecasting and anomaly detection is clear, yet realizing this potential requires a concerted effort across the scientific community, industry stakeholders, and policymakers. By addressing the challenges outlined in this review and harnessing the opportunities presented by emerging trends, we can look forward to a future where LLMs play a pivotal role in navigating the complexities of the modern world, driving insights and innovations that benefit all of society.

References

• [1]Alan M. Turing.Computing machinery and intelligence.Mind, 59(October):433–60, 1950.
• [2]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.BERT: Pre-training of deep bidirectional transformers for language understanding.In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
• [3]OpenAI, :, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mo Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford,Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez,Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, HenriRoussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, MarvinZhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph.GPT-4 technical report, 2023.
• [4]Matheus Gomes Sousa, Kenzo Sakiyama, Lucas de Souza Rodrigues, Pedro Henrique Moraes, Eraldo Rezende Fernandes, and Edson Takashi Matsubara.Bert for stock market sentiment analysis.In 2019 IEEE 31st international conference on tools with artificial intelligence (ICTAI), pages 1597–1601. IEEE, 2019.
• [5]Xin Jin and Yuchen Wang.Understand legal documents with contextualized large language models.arXiv preprint arXiv:2303.12135, 2023.
• [6]Lei Zhang, Mahsa Radnejad, and Andriy Miranskyy.Identifying flakiness in quantum programs.arXiv preprint arXiv:2302.03256, 2023.
• [7]Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.Deep learning.nature, 521(7553):436–444, 2015.
• [8]Jiashu Liao, Victor Sanchez, and Tanaya Guha.Self-supervised frontalization and rotation gan with random swap for pose-invariant face recognition.In 2022 IEEE International Conference on Image Processing (ICIP), pages 911–915. IEEE, 2022.
• [9]Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu.TEMPO: Prompt-based generative pre-trained transformer for time series forecasting.In The Twelfth International Conference on Learning Representations, 2024.
• [10]Song Chen and Hai Liao.BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model.Applied Artificial Intelligence, 36(1):2145642, December 2022.
• [11]Pranav K Nayak, Krishn V Kher, M Bharat Chandra, MV Rao, and Lei Zhang.Q-pac: Automated detection of quantum bug-fix patterns.arXiv preprint arXiv:2311.17705, 2023.
• [12]William Pourmajidi, Lei Zhang, John Steinbacher, Tony Erwin, and Andriy Miranskyy.Immutable log storage as a service on private and public blockchains.IEEE Transactions on Services Computing, 2021.
• [13]Penghui Zhang, Zhibo Sun, Sukwha Kyung, Hans Walter Behrens, Zion Leonahenahe Basque, Haehyun Cho, Adam Oest, Ruoyu Wang, Tiffany Bao, Yan Shosh*taishvili, et al.I’m spartacus, no, i’m spartacus: Proactively protecting users from phishing by intentionally triggering cloaking behavior.In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 3165–3179, 2022.
• [14]Zewen Wang, Lin Cai, Yahan Chen, Hongming Li, and Hanze Jia.The teaching design methods under educational psychology based on deep learning and artificial intelligence.Frontiers in Psychology, 12:711489, 2021.
• [15]Bryan Lim and Stefan Zohren.Time-series forecasting with deep learning: a survey.Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
• [16]Lei Zhang and Douglas G Down.Smva: a stable mean value analysis algorithm for closed systems with load-dependent queues.Systems Modeling: Methodologies and Tools, pages 11–28, 2019.
• [17]Jing Su, Suku Nair, and Leo Popokh.Optimal resource allocation in sdn/nfv-enabled networks via deep reinforcement learning.In 2022 IEEE Ninth International Conference on Communications and Networking (ComNet), pages 1–7. IEEE, 2022.
• [18]Jian Cao, Zhi Li, and Jian Li.Financial time series forecasting model based on ceemdan and lstm.Physica A: Statistical mechanics and its applications, 519:127–139, 2019.
• [19]Shun Liu, Kexin Wu, Chufeng Jiang, Bin Huang, and Danqing Ma.Financial time-series forecasting: Towards synergizing performance and interpretability within a hybrid machine learning approach.arXiv preprint arXiv:2401.00534, 2023.
• [20]Massimo Pacella and Gabriele Papadia.Evaluation of deep learning with long short-term memory networks for time series forecasting in supply chain management.Procedia CIRP, 99:604–609, 2021.
• [21]Scott L Zeger, Rafael Irizarry, and Roger D Peng.On time series analysis of public health and biomedical data.Annu. Rev. Public Health, 27:57–79, 2006.
• [22]Lei Zhang, Diego Lai, and Andriy V Miranskyy.The impact of position errors on crowd simulation.Simulation Modelling Practice and Theory, 90:45–63, 2019.
• [23]Weixia Dang, Biyu Zhou, Weigang Zhang, and Songlin Hu.Time Series Anomaly Detection Based on Language Model.In Proceedings of the Eleventh ACM International Conference on Future Energy Systems, E-Energy ’20, pages 544–547, New York, NY, USA, June 2020. Association for Computing Machinery.
• [24]Weixia Dang, Biyu Zhou, Lingwei Wei, Weigang Zhang, Ziang Yang, and Songlin Hu.TS-Bert: Time Series Anomaly Detection via Pre-training Model Bert.In Maciej Paszynski, Dieter Kranzlmüller, Valeria V. Krzhizhanovskaya, Jack J. Dongarra, and Peter M. A. Sloot, editors, Computational Science – ICCS 2021, Lecture Notes in Computer Science, pages 209–223, Cham, 2021. Springer International Publishing.
• [25]Mohiuddin Ahmed, Abdun Naser Mahmood, and Md Rafiqul Islam.A survey of anomaly detection techniques in financial domain.Future Generation Computer Systems, 55:278–288, 2016.
• [26]Jungwon Kim, Arlene Ong, and Richard E Overill.Design of an artificial immune system as a novel anomaly detector for combating financial fraud in the retail sector.In The 2003 Congress on Evolutionary Computation, 2003. CEC’03., volume 1, pages 405–412. IEEE, 2003.
• [27]Xin Jin, Charalampos Katsis, Fan Sang, Jiahao Sun, Ashish Kundu, and Ramana Kompella.Edge security: Challenges and issues.arXiv preprint arXiv:2206.07164, 2022.
• [28]Zhibo Sun, Carlos E Rubio-Medrano, Ziming Zhao, Tiffany Bao, Adam Doupé, and Gail-Joon Ahn.Understanding and predicting private interactions in underground forums.In Proceedings of the Ninth ACM Conference on Data and Application Security and Privacy, pages 303–314, 2019.
• [29]Kexin Wu and Kun Chi.Enhanced e-commerce customer engagement: A comprehensive three-tiered recommendation system.Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 2(3):348–359, 2023.
• [30]Zhibo Sun, Adam Oest, Penghui Zhang, Carlos Rubio-Medrano, Tiffany Bao, Ruoyu Wang, Ziming Zhao, Yan Shosh*taishvili, Adam Doupé, Gail-Joon Ahn, et al.Having your cake and eating it: An analysis of $\{$Concession-Abuse-as-a-Service$\}$.In 30th USENIX Security Symposium (USENIX Security 21), pages 4169–4186, 2021.
• [31]Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Eric Sun, and Yue Zhang.A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.arXiv preprint arXiv:2312.02003, 2023.
• [32]Xin Jin, Charalampos Katsis, Fan Sang, Jiahao Sun, Elisa Bertino, Ramana Rao Kompella, and Ashish Kundu.Prometheus: Infrastructure security posture analysis with ai-generated attack graphs.arXiv preprint arXiv:2312.13119, 2023.
• [33]Xinqi Dong, Bo Dang, Hengyi Zang, Shaojie Li, and Danqing Ma.The prediction trend of enterprise financial risk based on machine learning arima model.Journal of Theory and Practice of Engineering Science, 4(01):65–71, 2024.
• [34]Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019.
• [35]Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, andThomas Scialom.Llama 2: Open foundation and fine-tuned chat models.ArXiv, abs/2307.09288, 2023.
• [36]Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L’elio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.Mixtral of experts.ArXiv, abs/2401.04088, 2024.
• [37]Barbara A. Kitchenham.Systematic review in software engineering: Where we are and where we should be going.In Proceedings of the 2nd International Workshop on Evidential Assessment of Software Technologies, EAST ’12, pages 1–2, New York, NY, USA, 2012. Association for Computing Machinery.
• [38]Barbara Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman.Systematic literature reviews in software engineering – A systematic literature review.Information and Software Technology, 51(1):7–15, 2009.
• [39]Pearl Brereton, Barbara A Kitchenham, David Budgen, Mark Turner, and Mohamed Khalil.Lessons from applying the systematic literature review process within the software engineering domain.Journal of systems and software, 80(4):571–583, 2007.
• [40]Barbara Kitchenham, Rialette Pretorius, David Budgen, O Pearl Brereton, Mark Turner, Mahmood Niazi, and Stephen Linkman.Systematic literature reviews in software engineering–a tertiary study.Information and software technology, 52(8):792–805, 2010.
• [41]Aldeida Aleti, Barbora Buhnova, Lars Grunske, Anne Koziolek, and Indika Meedeniya.Software architecture optimization methods: A systematic literature review.IEEE Transactions on Software Engineering, 39(5):658–683, 2012.
• [42]Annalisa Cocchia.Smart and digital city: A systematic literature review.Smart city: How to create public and economic value with high technology in urban space, pages 13–43, 2014.
• [43]Chaomei Chen.Science mapping: A systematic review of the literature.Journal of data and information science, 2(2):1–40, 2017.
• [44]Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.ACM Computing Surveys, 55(9):1–35, 2023.
• [45]Nate Gruver, Marc Anton Finzi, Shikai Qiu, and Andrew Gordon Wilson.Large Language Models Are Zero-Shot Time Series Forecasters.In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
• [46]Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin.One Fits All: Power General Time Series Analysis by Pretrained LM.In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
• [47]Xiaoming Shi, Siqiao Xue, Kangrui Wang, Fan Zhou, James Y. Zhang, Jun Zhou, Chenhao Tan, and Hongyuan Mei.Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning.In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
• [48]Hao Xue and Flora D. Salim.PromptCast: A new prompt-based learning paradigm for time series forecasting.IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2023.
• [49]Yukyung Lee, Jina Kim, and Pilsung Kang.LAnoBERT: System log anomaly detection based on BERT masked language model.Applied Soft Computing, 146:110689, October 2023.
• [50]Hao Xue, Bhanu Prakash Voutharoja, and Flora D. Salim.Leveraging language foundation models for human mobility forecasting.In Proceedings of the 30th International Conference on Advances in Geographic Information Systems, SIGSPATIAL ’22, pages 1–9, New York, NY, USA, November 2022. Association for Computing Machinery.
• [51]Harold Ott, Jasmin Bogatinovski, Alexander Acker, Sasho Nedelkoski, and Odej Kao.Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models.In 2021 IEEE/ACM International Workshop on Cloud Intelligence (CloudIntelligence), pages 19–24, May 2021.
• [52]Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen.Time-LLM: Time series forecasting by reprogramming large language models.In The Twelfth International Conference on Learning Representations, 2024.
• [53]Qizhi Li, Xianyong Li, Yujia Song, Maolin Zhang, Longqi Chen, Gang Wang, and Yajun Du.Evaluating BERT on cloud-edge time series forecasting and sentiment analysis via prompt learning.In 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCit. IEEE, 2022.
• [54]Ting Zhang, Xin Huang, Wen Zhao, Shaohuang Bian, and Peng Du.LogPrompt: A Log-based Anomaly Detection Framework Using Prompts.In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, 2023.
• [55]KyoHoon Jin, JeongA Wi, EunJu Lee, ShinJin Kang, SooKyun Kim, and YoungBin Kim.TrafficBERT: Pre-trained model with large-scale data for long-range traffic flow forecasting.Expert Systems with Applications, 186:115738, December 2021.
• [56]Shaohan Huang, Yi Liu, Carol Fung, He Wang, Hailong Yang, and Zhongzhi Luan.Improving Log-Based Anomaly Detection by Pre-Training Hierarchical Transformers.IEEE Transactions on Computers, 72(9):2656–2667, September 2023.
• [57]Pranjal Gupta, Harsh*t Kumar, Debanjana Kar, Karan Bhukar, Pooja Aggarwal, and Prateeti Mohapatra.Learning Representations on Logs for AIOps.In 2023 IEEE 16th International Conference on Cloud Computing (CLOUD). IEEE, 2023.
• [58]Yangyi Shao, Wenbin Zhang, Peishun Liu, Ren Huyue, Ruichun Tang, Qilin Yin, and Qi Li.Log Anomaly Detection method based on BERT model optimization.In 2022 7th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA). IEEE, 2022.
• [59]Shiming He, Ying Lei, Ying Zhang, Kun Xie, and Pradip Kumar Sharma.Parameter-Efficient Log Anomaly Detection based on Pre-training model and LORA.In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2023.
• [60]Egil Karlsen, Rafael Copstein, Xiao Luo, Jeff Schwartzentruber, Bradley Niblett, Andrew Johnston, Malcolm I. Heywood, and Nur Zincir-Heywood.Exploring semantic vs. Syntactic features for unsupervised learning on application log files.In 2023 7th Cyber Security in Networking Conference (CSNet), pages 219–225, 2023.
• [61]Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long.SimMTM: A simple pre-training framework for masked time-series modeling.In Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
• [62]Caiping Hu, Xuekui Sun, Hua Dai, Hangchuan Zhang, and Haiqiang Liu.Research on log anomaly detection based on sentence-BERT.Electronicsweek, 12(3580), 2023.
• [63]Crispin Almodovar, Fariza Sabrina, Sarvnaz Karimi, and Salahuddin Azad.LogFiT: Log anomaly detection using fine-tuned language models.IEEE Transactions on Network and Service Management, pages 1–1, 2024.
• [64]Mingyang Zhang, Jianfei Chen, Jianyi Liu, Jingchu Wang, Rui Shi, and Hua Sheng.LogST: Log semi-supervised anomaly detection based on sentence-BERT.In 2022 7th International Conference on Signal and Image Processing (ICSIP), pages 356–361, 2022.
• [65]Van-Hoang Le and Hongyu Zhang.Log-based anomaly detection without log parsing.In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 492–504, 2021.
• [66]Shaohan Huang, Yi Liu, Carol Fung, Rong He, Yining Zhao, Hailong Yang, and Zhongzhi Luan.HitAnomaly: Hierarchical Transformers for Anomaly Detection in System Log.IEEE Transactions on Network and Service Management, 17(4):2064–2076, December 2020.
• [67]Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang.A review of recurrent neural networks: Lstm cells and network architectures.Neural computation, 31(7):1235–1270, 2019.
• [68]Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai, et al.Recent advances in convolutional neural networks.Pattern recognition, 77:354–377, 2018.
• [69]Danqing Ma, Bo Dang, Shaojie Li, Hengyi Zang, and Xinqi Dong.Implementation of computer vision technology based on artificial intelligence for medical image analysis.International Journal of Computer Science and Information Technology, 1(1):69–76, 2023.
• [70]Yi Zhu, Rong Wei, Ge Gao, Lian Ding, Xiaodong Zhang, Xiaoying Wang, and Jue Zhang.Fully automatic segmentation on prostate mr images based on cascaded fully convolution network.Journal of Magnetic Resonance Imaging, 49(4):1149–1156, 2019.
• [71]Ruixue Yuan, Hongming Li, Zhenyu Sun, and Huiwen Zhang.Application of graph convolutional network in the construction of knowledge graph for higher mathematics teaching.Sensors and Materials, 35(12):4269–4290, 2023.
• [72]Ajay Shrestha and Ausif Mahmood.Review of deep learning algorithms and architectures.IEEE access, 7:53040–53065, 2019.
• [73]François Gardères, Maryam Ziaeefard, Baptiste Abeloos, and Freddy Lecue.Conceptbert: Concept-aware representation for visual question answering.In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 489–498, 2020.
• [74]Clara Llorens Quintana, Siyu Chen, JUNHONG Lin, James G Fujimoto, Yan Li, and David Huang.Anterior topographic limbal demarcation with ultrawide-field oct.Investigative Ophthalmology & Visual Science, 63(7):1195–A0195, 2022.
• [75]Chenchen Ding, Hnin Thu Zar Aye, Win Pa Pa, Khin Thandar Nwet, Khin Mar Soe, Masao Utiyama, and Eiichiro Sumita.Towards burmese (myanmar) morphological analysis: Syllable-based tokenization and part-of-speech tagging.ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(1):1–34, 2019.
• [76]Tao Gui, Qi Zhang, Haoran Huang, Minlong Peng, and Xuan-Jing Huang.Part-of-speech tagging for twitter with adversarial neural networks.In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2411–2420, 2017.
• [77]Alan Ritter, Sam Clark, Oren Etzioni, et al.Named entity recognition in tweets: an experimental study.In Proceedings of the 2011 conference on empirical methods in natural language processing, pages 1524–1534, 2011.
• [78]Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li.A survey on deep learning for named entity recognition.IEEE Transactions on Knowledge and Data Engineering, 34(1):50–70, 2020.
• [79]Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al.Improving language understanding by generative pre-training, 2018.
• [80]Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.ACM Computing Surveys, 55(9):1–35, 2023.
• [81]Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon.Domain-specific language model pretraining for biomedical natural language processing.ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23, 2021.
• [82]Shanchan Wu and Yifan He.Enriching pre-trained language model with entity information for relation classification.In Proceedings of the 28th ACM international conference on information and knowledge management, pages 2361–2364, 2019.
• [83]Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, and Ziqing Yang.Pre-training with whole word masking for chinese bert.IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3504–3514, 2021.
• [84]Wei Bao, Hongshu Che, and Jiandong Zhang.Will_go at semeval-2020 task 3: An accurate model for predicting the (graded) effect of context in word similarity based on bert.In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 301–306, 2020.
• [85]Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan.Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems.In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
• [86]Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay.The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only.arXiv preprint arXiv:2306.01116, 2023.
• [87]Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.arXiv preprint arXiv:2301.12597, 2023.
• [88]Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al.Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models.PLoS digital health, 2(2):e0000198, 2023.
• [89]ji*ze Xiong, Mingyang Feng, Xiaosong Wang, Chufeng Jiang, Ning Zhang, and Zhiming Zhao.Decoding sentiments: Enhancing covid-19 tweet analysis through bert-rcnn fusion.Journal of Theory and Practice of Engineering Science, 4(01):86–93, 2024.
• [90]Kangrui Ruan, Cynthia He, Jiyang Wang, Xiaozhou Joey Zhou, Helian Feng, and Ali Kebarighotbi.S2e: Towards an end-to-end entity resolution solution from acoustic signal.In ICASSP 2024, 2024.
• [91]Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nie, and Tat-Seng Chua.Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation.In Proceedings of the 31st ACM International Conference on Multimedia, pages 643–654, 2023.
• [92]Zoie Zhao, Sophie Song, Bridget Duah, Jamie Macbeth, Scott Carter, Monica P Van, Nayeli Suseth Bravo, Matthew Klenk, Kate Sick, and Alexandre LS Filipowicz.More human than human: Llm-generated narratives outperform human-llm interleaved narratives.In Proceedings of the 15th Conference on Creativity and Cognition, pages 368–370, 2023.
• [93]Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin.Synthetic data generation with large language models for text classification: Potential and limitations.In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10443–10461, Singapore, December 2023. Association for Computational Linguistics.
• [94]Stavros Demetriadis and Yannis Dimitriadis.Conversational agents and language models that learn from human dialogues to support design thinking.In International Conference on Intelligent Tutoring Systems, pages 691–700. Springer, 2023.
• [95]Vojtěch Hudeček and Ondřej Dušek.Are large language models all you need for task-oriented dialogue?In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 216–228, 2023.
• [96]Zhi Jing, Yongye Su, Yikun Han, Bo Yuan, Haiyun Xu, Chunjiang Liu, Kehai Chen, and Min Zhang.When large language models meet vector databases: A survey, 2024.
• [97]Shoetsu Sato, Jin Sakuma, Naoki Yoshinaga, Masashi Toyoda, and Masaru Kitsuregawa.Vocabulary adaptation for domain adaptation in neural machine translation.In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4269–4279, 2020.
• [98]Ye Zhang, Mengran Zhu, Yulu Gong, and Rui Ding.Optimizing science question ranking through model and retrieval-augmented generation.International Journal of Computer Science and Information Technology, 1(1):124–130, 2023.
• [99]Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning.What does bert look at? an analysis of bert’s attention.arXiv preprint arXiv:1906.04341, 2019.
• [100]Ganesh Jawahar, Benoît Sagot, and Djamé Seddah.What does bert learn about the structure of language?In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, 2019.
• [101]Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou.Semantics-aware bert for language understanding.In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9628–9635, 2020.
• [102]Nils Reimers and Iryna Gurevych.Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019.
• [103]Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang.Evaluating commonsense in pre-trained language models.In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9733–9740, 2020.
• [104]Anastasia Chan.Gpt-3 and instructgpt: Technological dystopianism, utopianism, and “contextual” perspectives in ai ethics and industry.AI and Ethics, 3(1):53–64, 2023.
• [105]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020.
• [106]Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du.Explainability for large language models: A survey.ACM Transactions on Intelligent Systems and Technology, 2023.
• [107]Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui.A survey for in-context learning.arXiv preprint arXiv:2301.00234, 2022.
• [108]Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton.Big self-supervised models are strong semi-supervised learners.Advances in neural information processing systems, 33:22243–22255, 2020.
• [109]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
• [110]Tomáš Mikolov et al.Statistical language models based on neural networks.Presentation at Google, Mountain View, 2nd April, 80(26), 2012.
• [111]Adam Pauls and Dan Klein.Faster and smaller n-gram language models.In Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies, pages 258–267, 2011.
• [112]Antoine Tremblay and Benjamin V Tucker.The effects of n-gram probabilistic measures on the recognition and production of four-word sequences.The Mental Lexicon, 6(2):302–324, 2011.
• [113]Felix Wang, Tu-Thach Quach, Jason Wheeler, James B Aimone, and Conrad D James.Sparse coding for n-gram feature extraction and training for file fragment classification.IEEE Transactions on Information Forensics and Security, 13(10):2553–2562, 2018.
• [114]Khalil Sima’an and Luciano Buratto.Backoff parameter estimation for the dop model.In European Conference on Machine Learning, pages 373–384. Springer, 2003.
• [115]Alon Orlitsky and Ananda Theertha Suresh.Competitive distribution estimation: Why is good-turing good.Advances in Neural Information Processing Systems, 28, 2015.
• [116]Rebekah George Benjamin.Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty.Educational Psychology Review, 24:63–88, 2012.
• [117]Wei Xu and Alex Rudnicky.Can artificial neural networks learn language models?In Interspeech, 2000.
• [118]Yoshua Bengio, Réjean Ducharme, and Pascal Vincent.A neural probabilistic language model.Advances in neural information processing systems, 13, 2000.
• [119]Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin.A neural probabilistic language model.Journal of Machine Learning Research, 3:1137–1155, 2003.
• [120]Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur.Recurrent neural network based language model.In Interspeech, volume 2, pages 1045–1048. Makuhari, 2010.
• [121]Sepp Hochreiter and Jürgen Schmidhuber.Long short-term memory.Neural computation, 9(8):1735–1780, 1997.
• [122]Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.Learning phrase representations using RNN encoder-decoder for statistical machine translation.arXiv preprint arXiv:1406.1078, 2014.
• [123]Yann LeCun, Yoshua Bengio, et al.Convolutional networks for images, speech, and time series.Citeseer, 1995.
• [124]Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier.Language modeling with gated convolutional networks.In International Conference on Machine Learning, pages 933–941. PMLR, 2017.
• [125]Xavier Glorot and Yoshua Bengio.Understanding the difficulty of training deep feedforward neural networks.In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
• [126]Ilya Sutskever, James Martens, and Geoffrey E Hinton.Generating text with recurrent neural networks.In Proceedings of the 28th international conference on machine learning (ICML-11), pages 1017–1024, 2011.
• [127]Guanghui Wang, Yulu Gong, Mingwei Zhu, Jiaqiang Yuan, and Kuo Wei.Unveiling the future navigating next-generation ai frontiers and innovations in application.International Journal of Computer Science and Information Technology, 1(1):147–156, 2023.
• [128]Sepp Hochreiter.The vanishing gradient problem during learning recurrent neural nets and problem solutions.International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02):107–116, 1998.
• [129]Hongjie Niu, Hao Li, Jiufan Wang, Xiaonan Xu, and Huan Ji.Enhancing computer digital signal processing through the utilization of rnn sequence algorithms.International Journal of Computer Science and Information Technology, 1(1):60–68, 2023.
• [130]Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer.Deep contextualized word representations.In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
• [131]Justyna Sarzynska-Wawer, Aleksander Wawer, Aleksandra Pawlak, Julia Szymanowska, Izabela Stefaniak, Michal Jarkiewicz, and Lukasz Okruszek.Detecting formal thought disorder by deep contextualized word representations.Psychiatry Research, 304:114135, 2021.
• [132]Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.Neural machine translation by jointly learning to align and translate.arXiv preprint arXiv:1409.0473, 2014.
• [133]Minh-Thang Luong, Hieu Pham, and Christopher D Manning.Effective approaches to attention-based neural machine translation.arXiv preprint arXiv:1508.04025, 2015.
• [134]Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio.Attention-based models for speech recognition.Advances in neural information processing systems, 28, 2015.
• [135]Tianrui Liu, Changxin Xu, Yuxin Qiao, Chufeng Jiang, and Weisheng Chen.News recommendation with attention mechanism.arXiv preprint arXiv:2402.07422, 2024.
• [136]Xiaochen Wang, Tingsong Xiao, Jie Tan, Deqiang Ouyang, and Jie Shao.Mrmrp: multi-source review-based model for rating prediction.In Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24–27, 2020, Proceedings, Part II 25, pages 20–35. Springer, 2020.
• [137]Xiaochen Wang, Tingsong Xiao, and Jie Shao.Emrm: Enhanced multi-source review-based model for rating prediction.In Knowledge Science, Engineering and Management: 14th International Conference, KSEM 2021, Tokyo, Japan, August 14–16, 2021, Proceedings, Part III 14, pages 487–499. Springer, 2021.
• [138]Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.Bottom-up and top-down attention for image captioning and visual question answering.In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077–6086, 2018.
• [139]Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R Martin, Ming-Ming Cheng, and Shi-Min Hu.Attention mechanisms in computer vision: A survey.Computational visual media, 8(3):331–368, 2022.
• [140]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.Attention is all you need.In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
• [141]Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu.Eca-net: Efficient channel attention for deep convolutional neural networks.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11534–11542, 2020.
• [142]Maolin Li, Peng Ling, Shangsheng Wen, Xiandong Chen, and Fei Wen.Bubble-wave-mitigation algorithm and transformer-based neural network demodulator for water-air optical camera communications.IEEE Photonics Journal, 2023.
• [143]Xizhou Zhu, Dazhi Cheng, Zheng Zhang, Stephen Lin, and Jifeng Dai.An empirical study of spatial attention mechanisms in deep networks.In Proceedings of the IEEE/CVF international conference on computer vision, pages 6688–6697, 2019.
• [144]Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon.Cbam: Convolutional block attention module.In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
• [145]Xin Jin, Jonathan Larson, Weiwei Yang, and Zhiqiang Lin.Binary code summarization: Benchmarking chatgpt/gpt-4 and other large language models.arXiv preprint arXiv:2312.09601, 2023.
• [146]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018.
• [147]Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao.Multi-task deep neural networks for natural language understanding.In Anna Korhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4487–4496, Florence, Italy, July 2019. Association for Computational Linguistics.
• [148]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.Language models are few-shot learners.In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.
• [149]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe.Training language models to follow instructions with human feedback.In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 27730–27744. Curran Associates, Inc., 2022.
• [150]Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo.Gpt-4 passes the bar exam.Available at SSRN 4389233, 2023.
• [151]Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz.Capabilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023.
• [152]Yu Wang, Nedim Lipka, Ryan A Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr.Knowledge graph prompting for multi-document question answering.arXiv preprint arXiv:2308.11730, 2023.
• [153]Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang.Gpt understands, too.AI Open, 2023.
• [154]Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al.A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, 2023.
• [155]Steve Rathje, Dan-Mircea Mirea, Ilia Sucholutsky, Raja Marjieh, Claire Robertson, and Jay Joseph Van Bavel.GPT is an effective tool for multilingual psychological text analysis.PsyArXiv, May 2023.
• [156]AI21 Studio.Jurassic-2 models.https://docs.ai21.com/docs/jurassic-2-models, 2023.Accessed: 12th Feb 2024.
• [157]Anthropic.Making ai systems you can rely on.https://www.anthropic.com/company, 2023.Accessed: 12th Feb 2024.
• [158]Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al.Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022.
• [159]BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, Dragomir Radev, Eduardo González Ponferrada, Efrat Levkovizh, Ethan Kim, Eyal Bar Natan, Francesco De Toni, Gérard Dupont, Germán Kruszewski, Giada Pistilli, Hady Elsahar, Hamza Benyamina, Hieu Tran,Ian Yu, Idris Abdulmumin, Isaac Johnson, Itziar Gonzalez-Dios, Javier de la Rosa, Jenny Chim, Jesse Dodge, Jian Zhu, Jonathan Chang, Jörg Frohberg, Joseph Tobing, Joydeep Bhattacharjee, Khalid Almubarak, Kimbo Chen, Kyle Lo, Leandro Von Werra, Leon Weber, Long Phan, Loubna Ben allal, Ludovic Tanguy, Manan Dey, Manuel Romero Muñoz, Maraim Masoud, María Grandury, Mario Šaško, Max Huang, Maximin Coavoux, Mayank Singh, Mike Tian-Jian Jiang, Minh Chien Vu, Mohammad A. Jauhar, Mustafa Ghaleb, Nishant Subramani, Nora Kassner, Nurulaqilla Khamis, Olivier Nguyen, Omar Espejel, Ona de Gibert, Paulo Villegas, Peter Henderson, Pierre Colombo, Priscilla Amuok, Quentin Lhoest, Rheza Harliman, Rishi Bommasani, Roberto Luis López, Rui Ribeiro, Salomey Osei, Sampo Pyysalo, Sebastian Nagel, Shamik Bose, Shamsuddeen Hassan Muhammad, Shanya Sharma, Shayne Longpre, Somaieh Nikpoor, Stanislav Silberberg, Suhas Pai, Sydney Zink, Tiago Timponi Torrent, Timo Schick, Tristan Thrush, Valentin Danchev, Vassilina Nikoulina,Veronika Laippala, Violette Lepercq, Vrinda Prabhu, Zaid Alyafeai, Zeerak Talat, Arun Raja, Benjamin Heinzerling, Chenglei Si, Davut Emre Taşar, Elizabeth Salesky, Sabrina J. Mielke, Wilson Y. Lee, Abheesht Sharma, Andrea Santilli, Antoine Chaffin, Arnaud Stiegler, Debajyoti Datta, Eliza Szczechla, Gunjan Chhablani, Han Wang, Harsh*t Pandey, Hendrik Strobelt, Jason Alan Fries, Jos Rozen, Leo Gao, Lintang Sutawika, M Saiful Bari, Maged S. Al-shaibani, Matteo Manica, Nihal Nayak, Ryan Teehan, Samuel Albanie, Sheng Shen, Srulik Ben-David, Stephen H. Bach, Taewoon Kim, Tali Bers, Thibault Fevry, Trishala Neeraj, Urmish Thakker, Vikas Raunak, Xiangru Tang, Zheng-Xin Yong, Zhiqing Sun, Shaked Brody, Yallow Uri, Hadar Tojarieh, Adam Roberts, Hyung Won Chung, Jaesung Tae, Jason Phang, Ofir Press, Conglong Li, Deepak Narayanan, Hatim Bourfoune, Jared Casper, Jeff Rasley, Max Ryabinin, Mayank Mishra, Minjia Zhang, Mohammad Shoeybi, Myriam Peyrounette, Nicolas Patry, Nouamane Tazi, Omar Sanseviero, Patrick vonPlaten, Pierre Cornette, Pierre François Lavallée, Rémi Lacroix, Samyam Rajbhandari, Sanchit Gandhi, Shaden Smith, Stéphane Requena, Suraj Patil, Tim Dettmers, Ahmed Baruwa, Amanpreet Singh, Anastasia Cheveleva, Anne-Laure Ligozat, Arjun Subramonian, Aurélie Névéol, Charles Lovering, Dan Garrette, Deepak Tunuguntla, Ehud Reiter, Ekaterina Taktasheva, Ekaterina Voloshina, Eli Bogdanov, Genta Indra Winata, Hailey Schoelkopf, Jan-Christoph Kalo, Jekaterina Novikova, Jessica Zosa Forde, Jordan Clive, Jungo Kasai, Ken Kawamura, Liam Hazan, Marine Carpuat, Miruna Clinciu, Najoung Kim, Newton Cheng, Oleg Serikov, Omer Antverg, Oskar van der Wal, Rui Zhang, Ruochen Zhang, Sebastian Gehrmann, Shachar Mirkin, Shani Pais, Tatiana Shavrina, Thomas Scialom, Tian Yun, Tomasz Limisiewicz, Verena Rieser, Vitaly Protasov, Vladislav Mikhailov, Yada Pruksachatkun, Yonatan Belinkov, Zachary Bamberger, Zdeněk Kasner, Alice Rueda, Amanda Pestana, Amir Feizpour, Ammar Khan, Amy Faranak, Ana Santos, Anthony Hevia, AntigonaUnldreaj, Arash Aghagol, Arezoo Abdollahi, Aycha Tammour, Azadeh HajiHosseini, Bahareh Behroozi, Benjamin Ajibade, Bharat Saxena, Carlos Muñoz Ferrandis, Daniel McDuff, Danish Contractor, David Lansky, Davis David, Douwe Kiela, Duong A. Nguyen, Edward Tan, Emi Baylor, Ezinwanne Ozoani, Fatima Mirza, Frankline Ononiwu, Habib Rezanejad, Hessie Jones, Indrani Bhattacharya, Irene Solaiman, Irina Sedenko, Isar Nejadgholi, Jesse Passmore, Josh Seltzer, Julio Bonis Sanz, Livia Dutra, Mairon Samagaio, Maraim Elbadri, Margot Mieskes, Marissa Gerchick, Martha Akinlolu, Michael McKenna, Mike Qiu, Muhammed Ghauri, Mykola Burynok, Nafis Abrar, Nazneen Rajani, Nour Elkott, Nour Fahmy, Olanrewaju Samuel, Ran An, Rasmus Kromann, Ryan Hao, Samira Alizadeh, Sarmad Shubber, Silas Wang, Sourav Roy, Sylvain Viguier, Thanh Le, Tobi Oyebade, Trieu Le, Yoyo Yang, Zach Nguyen, Abhinav Ramesh Kashyap, Alfredo Palasciano, Alison Callahan, Anima Shukla, Antonio Miranda-Escalada, Ayush Singh, Benjamin Beilharz, Bo Wang, Caio Brito,Chenxi Zhou, Chirag Jain, Chuxin Xu, Clémentine Fourrier, Daniel León Periñán, Daniel Molano, Dian Yu, Enrique Manjavacas, Fabio Barth, Florian Fuhrimann, Gabriel Altay, Giyaseddin Bayrak, Gully Burns, Helena U. Vrabec, Imane Bello, Ishani Dash, Jihyun Kang, John Giorgi, Jonas Golde, Jose David Posada, Karthik Rangasai Sivaraman, Lokesh Bulchandani, Lu Liu, Luisa Shinzato, Madeleine Hahn de Bykhovetz, Maiko Takeuchi, Marc Pàmies, Maria A Castillo, Marianna Nezhurina, Mario Sänger, Matthias Samwald, Michael Cullan, Michael Weinberg, Michiel De Wolf, Mina Mihaljcic, Minna Liu, Moritz Freidank, Myungsun Kang, Natasha Seelam, Nathan Dahlberg, Nicholas Michio Broad, Nikolaus Muellner, Pascale Fung, Patrick Haller, Ramya Chandrasekhar, Renata Eisenberg, Robert Martin, Rodrigo Canalli, Rosaline Su, Ruisi Su, Samuel Cahyawijaya, Samuele Garda, Shlok S Deshmukh, Shubhanshu Mishra, Sid Kiblawi, Simon Ott, Sinee Sang-aroonsiri, Srishti Kumar, Stefan Schweter, Sushil Bharati, Tanmay Laud, Théo Gigant, TomoyaKainuma, Wojciech Kusa, Yanis Labrak, Yash Shailesh Bajaj, Yash Venkatraman, Yifan Xu, Yingxin Xu, Yu Xu, Zhe Tan, Zhongli Xie, Zifan Ye, Mathilde Bras, Younes Belkada, and Thomas Wolf.Bloom: A 176b-parameter open-access multilingual language model, 2023.
• [160]Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.Huggingface’s transformers: State-of-the-art natural language processing, 2020.
• [161]Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.Transformers: State-of-the-art natural language processing.In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online, October 2020. Association for Computational Linguistics.
• [162]Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh.Autoprompt: Eliciting knowledge from language models with automatically generated prompts.arXiv preprint arXiv:2010.15980, 2020.
• [163]Xingchen Song, Di Wu, Binbin Zhang, Zhendong Peng, Bo Dang, Fuping Pan, and Zhiyong Wu.Zeroprompt: Streaming acoustic encoders are zero-shot masked lms.arXiv preprint arXiv:2305.10649, 2023.
• [164]Tianyu Gao, Adam Fisch, and Danqi Chen.Making pre-trained language models better few-shot learners.arXiv preprint arXiv:2012.15723, 2020.
• [165]Gamaleldin F Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein.Adversarial reprogramming of neural networks.arXiv preprint arXiv:1806.11146, 2018.
• [166]Pin-Yu Chen.Model reprogramming: Resource-efficient cross-domain machine learning.arXiv preprint arXiv:2202.10629, 2022.
• [167]Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu, and Shiyu Chang.Fairness reprogramming.Advances in Neural Information Processing Systems, 35:34347–34362, 2022.
• [168]Peyman Bateni, Raghav Goyal, Vaden Masrani, Frank Wood, and Leonid Sigal.Improved few-shot visual classification.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14493–14502, 2020.
• [169]Yuchen Wang, Xin Jin, and Carlos E. Castro.Accelerating the characterization of dynamic dna origami devices with deep neural networks.Scientific Reports, 13, 2023.
• [170]Yunxiao Deng, Carl Kesselman, Suvrajeet Sen, and Jiajun Xu.Computational operations research exchange (core): A cyber-infrastructure for analytics.In 2019 Winter Simulation Conference (WSC), pages 3447–3456. IEEE, 2019.
• [171]Jiajun Xu and Suvrajeet Sen.Decision intelligence for nationwide ventilator allocation during the covid-19 pandemic.SN Computer Science, 2(6):423, 2021.
• [172]Jiajun Xu and Suvrajeet Sen.Ensemble variance reduction methods for stochastic mixed-integer programming and their application to the stochastic facility location problem.INFORMS Journal on Computing, 2023.
• [173]Jiajun Xu and Suvrajeet Sen.Compromise policy for multi-stage stochastic linear programming: Variance and bias reduction.Computers & Operations Research, 153:106132, 2023.
• [174]Jianmo Ni, Jiacheng Li, and Julian McAuley.Justifying recommendations using distantly-labeled reviews and fine-grained aspects.In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 188–197, Hong Kong, China, November 2019. Association for Computational Linguistics.
• [175]Julien Herzen, Francesco Lässig, Samuele Giuliano Piazzetta, Thomas Neuer, Léo Tafti, Guillaume Raille, Tomas Van Pottelbergh, Marek Pasieka, Andrzej Skrodzki, Nicolas Huguenin, Maxime Dumonal, Jan Kościsz, Dennis Bader, Frédérick Gusset, Mounir Benheddi, Camila Williamson, Michal Kosinski, Matej Petrik, and Gaël Grosch.Darts: User-friendly modern machine learning for time series.The Journal of Machine Learning Research, 23(1):124:5442–124:5447, January 2022.
• [176]Artur Trindade.ElectricityLoadDiagrams20112014.UCI Machine Learning Repository, 2015.
• [177]Elizabeth Boschee, Jennifer Lautenschlager, Sean O’Brien, Steve Shellman, James Starz, and Michael Ward.ICEWS coded event data, 2015.
• [178]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang.Informer: Beyond efficient transformer for long sequence time-series forecasting.In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11106–11115, 2021.
• [179]Spyros Makridakis and Michèle Hibon.The M3-Competition: Results, conclusions and implications.International Journal of Forecasting, 16(4):451–476, 2000.
• [180]Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos.The M4 Competition: 100,000 time series and 61 forecasting methods.International Journal of Forecasting, 36(1):54–74, 2020.
• [181]Raksh*tha Wathsadini Godahewa, Christoph Bergmeir, Geoffrey I. Webb, Rob Hyndman, and Pablo Montero-Manso.Monash time series forecasting archive.In Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
• [182]Adam Oliner and Jon Stearley.What supercomputers say: A study of five system logs.In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07), pages 575–584, 2007.
• [183]Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan.Detecting large-scale system problems by mining console logs.In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP ’09, pages 117–132, New York, NY, USA, 2009. Association for Computing Machinery.
• [184]Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar.DeepLog: Anomaly detection and diagnosis from system logs through deep learning.In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, pages 1285–1298, New York, NY, USA, 2017. Association for Computing Machinery.
• [185]Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei.Robust anomaly detection for multivariate time series through stochastic recurrent neural network.In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages 2828–2837, New York, NY, USA, 2019. Association for Computing Machinery.
• [186]Jinghua Sheng.An augmentable domain-specific models for financial analysis.In 2023 16th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pages 1–4, 2023.
• [187]Xin Jin, Kexin Pei, Jun Yeon Won, and Zhiqiang Lin.Symlm: Predicting function names in stripped binaries via context-sensitive execution-aware code embeddings.In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1631–1645, 2022.
• [188]Xin Jin, Sunil Manandhar, Kaushal Kafle, Zhiqiang Lin, and Adwait Nadkarni.Understanding iot security from a market-scale perspective.In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1615–1629, 2022.
• [189]James A Hanley and Barbara J McNeil.The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology, 143(1):29–36, 1982.
• [190]I. de Zarzà, J. de Curtò, Gemma Roig, and Carlos T. Calafate.LLM Multimodal Traffic Accident Forecasting.Sensors, 23(22):9225, January 2023.
• [191]Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha.Transformer hawkes process.In International conference on machine learning, pages 11692–11702. PMLR, 2020.
• [192]Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz.Self-attentive hawkes process.In International conference on machine learning, pages 11183–11193. PMLR, 2020.
• [193]Chenghao Yang, Hongyuan Mei, and Jason Eisner.Transformer embeddings of irregularly spaced events and their participants.In Proceedings of the Tenth International Conference on Learning Representations (ICLR), 2022.
• [194]Siqiao Xue, Yan Wang, Zhixuan Chu, Xiaoming Shi, Caigao JIANG, Hongyan Hao, Gangwei Jiang, Xiaoyun Feng, James Y Zhang, and JUN ZHOU.Prompt-augmented temporal point process for streaming event sequence.In Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
• [195]Nishanth Nakshatri, Siyi Liu, Sihao Chen, Dan Roth, Dan Goldwasser, and Daniel Hopkins.Using llm for improving key event discovery: Temporal-guided news stream clustering with event summaries.In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4162–4173, 2023.
• [196]Kangrui Ruan and Xuan Di.Learning human driving behaviors with sequential causal imitation learning.In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4583–4592, 2022.
• [197]Zhaobin Mo, Wangzhi Li, Yongjie Fu, Kangrui Ruan, and Xuan Di.Cvlight: Decentralized learning for adaptive traffic signal control with connected vehicles.Transportation research part C: emerging technologies, 141:103728, 2022.
• [198]Adam Szirmai.The dynamics of socio-economic development: an introduction.Cambridge University Press, 2005.
• [199]Tingsong Xiao, Lu Zeng, Xiaoshuang Shi, Xiaofeng Zhu, and Guorong Wu.Dual-graph learning convolutional networks for interpretable alzheimer’s disease diagnosis.In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 406–415. Springer, 2022.
• [200]Lu Zeng, Hengxin Li, Tingsong Xiao, Fumin Shen, and Zhi Zhong.Graph convolutional network with sample and feature weights for alzheimer’s disease diagnosis.Information Processing & Management, 59(4):102952, 2022.
• [201]Lavender Yao Jiang, Xujin Chris Liu, Nima Pour Nejatian, Mustafa Nasir-Moin, Duo Wang, Anas Abidin, Kevin Eaton, Howard Antony Riina, Ilya Laufer, Paawan Punjabi, Madeline Miceli, Nora C. Kim, Cordelia Orillac, Zane Schnurman, Christopher Livia, Hannah Weiss, David Kurland, Sean Neifert, Yosef Dastagirzada, Douglas Kondziolka, Alexander T. M. Cheung, Grace Yang, Ming Cao, Mona Flores, Anthony B. Costa, Yindalon Aphinyanaphongs, Kyunghyun Cho, and Eric Karl Oermann.Health system-scale language models are all-purpose prediction engines.Nature, 619(7969):357–362, July 2023.
• [202]Van-Hoang Le and Hongyu Zhang.Log Parsing: How Far Can ChatGPT Go?In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2023.
• [203]Mukhsimbayev Bobur, Kuralbayev Aibek, Bekbaganbetov Abay, and Fuad Hajiyev.Anomaly Detection Between Judicial Text-Based Documents.In 2020 IEEE 14th International Conference on Application of Information and Communication Technologies (AICT). IEEE, 2020.
• [204]Sarda Komal, Namrud Zakeya, Rouf Raphael, Ahuja Harit, Rasolroveicy Mohammadreza, Litoiu Marin, Shwartz Larisa, and Watts Ian.ADARMA auto-detection and auto-remediation of microservice anomalies by leveraging large language models.In Proceedings of the 33rd Annual International Conference on Computer Science and Software Engineering, CASCON ’23, pages 200–205, USA, 2023. IBM Corp.
• [205]Pitikorn Khlaisamniang, Prachaya Khomduean, Kriangkrai Saetan, and Supasin Wonglapsuwan.Generative AI for self-healing systems.In 2023 18th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP), pages 1–6, 2023.
• [206]Lei Zhang, Sean Howard, Tom Montpool, Jessica Moore, Krittika Mahajan, and Andriy Miranskyy.Automated data validation: An industrial experience report.Journal of Systems and Software, 197:111573, 2023.
• [207]Lei Zhang, Andriy Miranskyy, Walid Rjaibi, Greg Stager, Michael Gray, and John Peck.Making existing software quantum safe: A case study on ibm db2.Information and Software Technology, 161:107249, 2023.
• [208]Zhibo Sun, Hong Zhang, Zixia Liu, Chen Xu, and Liqiang Wang.Migrating gis big data computing from hadoop to spark: an exemplary study using twitter.In 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pages 351–358. IEEE, 2016.
• [209]Zhuoyan Li, Zhuoran Lu, and Ming Yin.Modeling human trust and reliance in ai-assisted decision making: a markovian approach.In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 2023.
• [210]Hongyuan Dong, Jiaqing Xie, Zhi Jing, and Dexin Ren.Variational autoencoder for anti-cancer drug response prediction, 2021.
• [211]Ye Zhang, Yulu Gong, Dongji Cui, Xinrui Li, and Xinyu Shen.Deepgi: An automated approach for gastrointestinal tract segmentation in mri scans.arXiv preprint arXiv:2401.15354, 2024.
• [212]Junhong Lin.Deep-learning Enabled Accurate Bruch’s Membrane Segmentation in Ultrahigh-Resolution Spectral Domain and Ultrahigh-Speed Swept Source Optical Coherence Tomography.PhD thesis, Massachusetts Institute of Technology, 2022.
• [213]Siyu Chen, Benjamin Potsaid, Junhong Lin, Yunchan Hwang, Eric Moult, Yan Li, David Huang, and James G Fujimoto.High-speed, long-range swept-source optical coherence tomography for the anterior segment of the eye.Investigative Ophthalmology & Visual Science, 62(11):75–75, 2021.
• [214]Siyu Chen, Benjamin Potsaid, Yan Li, Junhong Lin, Yunchan Hwang, Eric M Moult, Jason Zhang, David Huang, and James G Fujimoto.High speed, long range, deep penetration swept source oct for structural and angiographic imaging of the anterior eye.Scientific reports, 12(1):992, 2022.
• [215]Siyu Chen, Omar Abu-Qamar, Deepayan Kar, Jeffrey D Messinger, Yunchan Hwang, Eric M Moult, Junhong Lin, Caroline R Baumal, Andre Witkin, Michelle C Liang, et al.Ultrahigh resolution oct markers of normal aging and early age-related macular degeneration.Ophthalmology Science, 3(3):100277, 2023.
• [216]Pujin Cheng, Li Lin, Junyan Lyu, Yijin Huang, Wenhan Luo, and Xiaoying Tang.Prior: Prototype representation joint learning from medical images and reports.In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
• [217]Yongfei Liu, Haoyu Yang, and Chenwei Wu.Unveiling patterns: A study on semi-supervised classification of strip surface defects.IEEE Access, 11:119933–119946, 2023.
• [218]Jing Su, Suku Nair, and Leo Popokh.Edgegym: A reinforcement learning environment for constraint-aware nfv resource allocation.In 2023 IEEE 2nd International Conference on AI in Cybersecurity (ICAIC), pages 1–7. IEEE, 2023.
• [219]Junhong Lin, Bang Wang, Guang Yang, and Mu Zhou.Indoor localization based on weighted surfacing from crowdsourced samples.Sensors, 18(9):2990, 2018.
• [220]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.Learning transferable visual models from natural language supervision.In International conference on machine learning, pages 8748–8763. PMLR, 2021.
• [221]Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick.Segment anything.arXiv:2304.02643, 2023.
• [222]Jiashu Liao, Tanaya Guha, and Victor Sanchez.Self-supervised random mask attention gan in tackling pose-invariant face recognition.Available at SSRN 4583223, 2023.
• [223]Yongfei Liu, Haoyu Yang, and Chenwei Wu.Unveiling patterns: A study on semi-supervised classification of strip surface defects.IEEE Access, 11:119933–119946, 2023.
• [224]Zhongkai Shangguan, Yue Zhao, Wei Fan, and Zhehan Cao.Dog image generation using deep convolutional generative adversarial networks.In 2020 5th international conference on universal village (UV), pages 1–6. IEEE, 2020.
• [225]Lei Zhang and Douglas G Down.Approximate mean value analysis for multi-core systems.In 2015 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), pages 1–8. IEEE, 2015.
• [226]Hengyi Zang.Precision calibration of industrial 3d scanners: An ai-enhanced approach for improved measurement accuracy.Global Academic Frontiers, 2(1):27–37, 2024.
• [227]Carl Herickhoff, Junhong Lin, and Jeremy Dahl.Low-cost sensor-enabled freehand 3d ultrasound.In 2019 IEEE International Ultrasonics Symposium (IUS), pages 498–501. IEEE, 2019.
• [228]Tianrui Liu, Changxin Xu, Yuxin Qiao, Chufeng Jiang, and Jiqiang Yu.Particle filter slam for vehicle localization.arXiv preprint arXiv:2402.07429, 2024.
• [229]Chun-Wei Chiang, Zhuoran Lu, Zhuoyan Li, and Ming Yin.Are two heads better than one in ai-assisted decision making? comparing the behavior and performance of groups and individuals in human-ai collaborative recidivism risk assessment.In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2023.
• [230]Fanghao Ni, Hengyi Zang, and Yuxin Qiao.Smartfix: Leveraging machine learning for proactive equipment maintenance in industry 4.0.In The 2nd International scientific and practical conference “Innovations in education: prospects and challenges of today”(January 16-19, 2024) Sofia, Bulgaria. International Science Group. 2024. 389 p., page 313, 2024.
• [231]Yuxin Qiao, Jiajun Jin, Fanghao Ni, Jiqiang Yu, and Weisheng Chen.Application of machine learning in financial risk early warning and regional prevention and control: A systematic analysis based on shap.WORLD TRENDS, REALITIES AND ACCOMPANYING PROBLEMS OF DEVELOPMENT, page 331, 2023.
• [232]Leo Popokh, Jing Su, Suku Nair, and Eli Olinick.Illumicore: Optimization modeling and implementation for efficient vnf placement.In 2021 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pages 1–7. IEEE, 2021.
• [233]Kuo Wei, Hengyi Zang, Yiming Pan, Guanghui Wang, and Zepeng Shen.Strategic application of ai intelligent algorithm in network threat detection and defense.Journal of Theory and Practice of Engineering Science, 4(01):49–57, 2024.
• [234]Jing Hu, Xincheng Wang, Ziheng Liao, and Tingsong Xiao.M-gcn: Multi-scale graph convolutional network for 3d point cloud classification.In 2023 IEEE International Conference on Multimedia and Expo (ICME), pages 924–929. IEEE, 2023.
• [235]Zhibo Sun, Ashish Dhital, Nattaya Areejitkasem, Neera Pradhan, and Amy Banic.Effects on performance of analytical tools for visually demanding tasks through direct and indirect touch interaction in an immersive visualization.In 2014 International Conference on Virtual Reality and Visualization, pages 186–193. IEEE, 2014.
• [236]Zhuoran Lu, Zhuoyan Li, Chun-Wei Chiang, and Ming Yin.Strategic adversarial attacks in ai-assisted decision making to reduce human trust and reliance.In Edith Elkind, editor, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 3020–3028. International Joint Conferences on Artificial Intelligence Organization, 8 2023.Main Track.
• [237]Hengyi Zang, Shaojie Li, Xinqi Dong, Danqing Ma, and Bo Dang.Evaluating the social impact of ai in manufacturing: A methodological framework for ethical production.Academic Journal of Sociology and Management, 2(1):21–25, 2024.
• [238]Zihe Zheng, Zhongkai Shangguan, and Jiebo Luo.What makes a turing award winner?In Social, Cultural, and Behavioral Modeling: 14th International Conference, SBP-BRiMS 2021, Virtual Event, July 6–9, 2021, Proceedings 14, pages 310–320. Springer, 2021.
• [239]Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen.A survey of large language models, 2023.
• [240]Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu.Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond, 2023.
• [241]Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie.A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, January 2024.
• [242]Tyler A. Chang and Benjamin K. Bergen.Language model behavior: A comprehensive survey, 2023.
• [243]Jie Huang and Kevin Chen-Chuan Chang.Towards reasoning in large language models: A survey.In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, pages 1049–1065, Toronto, Canada, July 2023. Association for Computational Linguistics.