Does Azure MSI support accessing Graph API?

I have a VM created in Azure with MSI (Managed Service Identity) enabled and I also grant the contributor role of my subscription to the VM, so from this VM, I am able to call "localhost:50342" to get the access token and then use Azure Resource Manager API (endpoint: management.azure.com) to access Azure resources. Now when I tried to use the same way to access Azure Graph API (endpoint: graph.microsoft.com), I kept getting "[code] => Authorization_RequestDenied [value] => Insufficient privileges to complete the operation". So how am I able to grant permissions to the VM to access Azure Graph API when MSI is enabled? Thank you very much!