While following the "Tutorial: Azure Active Directory integration with Amazon Web Services (AWS)", I discovered that the step 2 of "Configure Azure AD single sign-on" section - doesn't work. Specifically the choice of "SAML" is never saved. Instead, it always reverts back to disabled (there is no "Save" button either). If you were to follow the guide as is, when you'd click "Test", you would be greeted by this error message: "This functionality is not enabled or not available".
The work-around is to switch to do the relevant setup, then switch to the "old experience" and click save. That enables SAML finally.