Abstract
ChatGPT and other state-of-the-art large language models (LLMs) are rapidlytransforming multiple fields, offering powerful tools for a wide range ofapplications. These models, commonly trained on vast datasets, exhibithuman-like text generation capabilities, making them useful for research taskssuch as ideation, literature review, coding, drafting, and outreach. Weconducted a study involving 13 astronomers at different career stages andresearch fields to explore LLM applications across diverse tasks over severalmonths and to evaluate their performance in research-related activities. Thiswork was accompanied by an anonymous survey assessing participants' experiencesand attitudes towards LLMs. We provide a detailed analysis of the tasksattempted and the survey answers, along with specific output examples. Ourfindings highlight both the potential and limitations of LLMs in supportingresearch while also addressing general and research-specific ethicalconsiderations. We conclude with a series of recommendations, emphasizing theneed for researchers to complement LLMs with critical thinking and domainexpertise, ensuring these tools serve as aids rather than substitutes forrigorous scientific inquiry.