GPT-4 and other large language models are being used in various areas of daily life, including browsing, voice assistants, and coding tools. AI systems have the potential to benefit society by reducing costs in fields like healthcare and law, improving accessibility and learning opportunities, and aiding in addressing climate change and pandemic preparedness. However, we should not expect them to do so by default. In fact, in the absence of proper governance, we should expect increasingly advanced AI systems to enable severe harms to individuals and society. This talk emphasizes the need for accountability measures such as auditing, external scrutiny, and regulation to ensure that AI development benefits and does not harm all of humanity.