See this post for an overview of Private Google Access and Cloud NAT.

Subnet versus VPC Level

One key difference is that Private Google Access is defined at the subnet level. This is not to be confused with Private Service access, which is defined at a VPC Level and applies to ALL subnets within the VPC.

Apart from the Subnet/VPC application level, Another way to differentiate these services is to understand WHO they target (the client VMs). 

For Private google Access , typically we are talking about Compute Instances WITHOUT external IPs.  If you private IP addresses that need access to PUBLIC APIs (Google APIs or your own hosted APIs), think of Private Google Access.  (We can also use Private Google Access for on premises hosts - which can or cannot have External IP addresses.)

For Private Service Connect , the clients are again GCP VMs - this time even external IPs are allowed. Here, the two subcategories of the service are a) Are you connecting to an internal Load  balancer or b) Are you connecting to Private Endpoint (created by Private google access) in your VPC.

Why not just use Peered VPCs?

Private Service Endpoints cannot be accessed from peered VPCs, by design.

Real world example?

Say you have a DEV subnet and a PROD subnet in your VPC. Say your company requires that PROD instances have NO internet facing instances (no Public IPs on instances). However, these instances need to access your cloud storage service. In this case, you would enable Private Google Access on the PROD subnet. This will let those PROD instances access cloud storage without going over the internet.

Can you host your own APIs and connect - using Private Service Connect?

Yes-  both Google APIs and your own hosted APIs can be accessed via endpoints (PSC endpoints).  You would add your service endpoint to GCP's service directory (a special service for this purpose).

How does Private Service Connect even work?

Essentially, through reserved Private IPs (RFC1918) that get reserved within your VPC (when you set up Private service connect). You end up creating a 'Private Service Endpoint' for each service. Sending traffic to that private IP routes the traffic to the Google PaaS Service. You can even set up custom DNS name (private zone) to send traffic to a name rather than an IP.

For Private Service Connect , the clients are again GCP VMs - with or without external IPs. Here, the two subcategories of the service are a) Are you connecting to an internal Load  balancer or b) Are you connecting to Private Endpoint in your VPC.

Option Clients Connection Supported services Usage
Connecting to Google APIs
Private Service Connect for Google APIs
Google Cloud resources with or without external IP addresses, and on-premises systems. Connect to a Private Service Connect endpoint in your VPC network, which forwards requests to Google APIs and services. Supports most Google APIs and services. Connect to Google APIs and services using an endpoint in your VPC network. Google Cloud and on-premises resources don't need an external IP addresses.
Private Service Connect for Google APIs with consumer HTTP(S) service controls (Preview)
Google Cloud resources with or without external IP addresses, and on-premises systems. Connect to an internal HTTP(S) load balancer in your VPC network, which forwards requests to Google APIs and services. Supports selected regional Google APIs and services. Connect to regional Google APIs and services using an internal HTTP(S) load balancer in your VPC network. Google Cloud and on-premises resources don't need an external IP addresses.
Private Google Access
Google Cloud resources without external IP addresses. Connect to the standard external IP addresses or Private Google Access domains and VIPs for Google APIs and services through the VPC network's default internet gateway. Supports most Google APIs and services. Use this option to connect to Google APIs and services without giving your Google Cloud resources external IP addresses.
Private Google Access for on-premises hosts
On-premises hosts with or without external IP addresses. Connect to Google APIs and services, from your on-premises network, through a Cloud VPN tunnel or Cloud Interconnect by using one of the Private Google Access-specific domains and VIPs. The Google services that you can access depend on which Private Google Access-specific domain you use. Use this option to connect to Google APIs and services through a VPC network. This method doesn't require your on-premises hosts to have external IP addresses.