Okay, something new appears to have snuck in.
Normally naval units can only bombard if they are supporting land units or they are returning fire on land units which are bombarding them. They cannot just sail in somewhere and blast away at enemy units in adjacent land regions.
For land units to be able to bombard naval units they
1. have to have canons that have enough range to reach those naval units (I believe on rivers the range is shorter than on coastal region)
2. they must either a) be in an old style fort or b) be at entrenched level 3 or higher and have the bombardment button pressed.
Beyond this there is something called the "double adjacency rule". This is a rule of thumb and not an absolute rule. The rule says, for land units meeting 1. and 2. the naval units must pass from one river or coastal region adjacent to the land units into another river or coastal region adjacent to the land units, thus the "double adjacency rule".
A good example of this is Fort Gaines and Fort Morgan, south of Mobile, AL. Fort Gaines has Harbor Exit Points Jackson Shore, Mobile Bay and Falmouth Beach. Fort Morgan has Harbor Exit Points Mobile Bay and Falmouth Beach.
Naval units sailing from Falmouth Beach into Mobile Bay meet the requirements of the double adjacency rule and will likely (practically always) be bombarded from Coastal Artillery batteries in Fort Gaines and Fort Morgan.
As I said, the double adjacency rule is a rule of thumb, not an absolute rule. This means that naval units moving from Biloxi Bay into Jackson Shore can be bombarded from Fort Gaines, but it is much less likely. The same goes for units leaving such an adjacent river or coastal region. As the CSA if you've ever sent naval units past Fort Monroe - and you probably have - you know that you either get a message saying the you successfully bypassed the Fort Monroe batteries or that you were bombarded by them.
What has changed is that now if your naval units have their bombardment button pressed and they are in a river or coastal region with enemy land units meeting 1. and 2. above, bombardment will take place. In fact you can sail your naval units into such a region in defensive posture and using 'avoid combat' to spare cohesion and in the next turn click their bombardment button, and bombardment will take place, always, for as long at the bombardment button is pressed.