AuroraGPT is an ambitious pilot project to develop and improve methodologies that the science community can use to produce end-to-end pre-trained, and instruct-tuned and aligned models, as will be important for developing the type of general- purpose scientific foundation models advocated for in DOE’s AI for Science planning process, and envisioned by the DOE FASST initiative. AuroraGPT aims to enhance the development and understanding of foundation models for science by exploring larger scientific corpora, more diverse types of data, and examining the role of modeling choices on the scientific reasoning tasks. The project’s outcome has the potential to improve significantly how science is conducted by fostering a deeper integration of AI capabilities into research workflows. The AuroraGPT project will build a series of tools that assist researchers in making more informed and efficient scientific discoveries, greatly impacting the scientific landscape. The main tasks in the project include collecting and refining large-scale scientific datasets; building models at 8 billion to 400 billion or more parameter scales using general texts, code, and specific scientific data, and evaluating their performance on the Aurora and Polaris supercomputers; refining the models for deployment and introducing post-processing techniques such as instruct tuning and Reinforcement Learning for aligned chat-based interfaces; and evaluating the effectiveness of the models on scientific tasks. AuroraGPT offers a transformative opportunity to leverage AI for scientific discovery, potentially redefining problem-solving across various domains critical to the DOE’s mission.