Playing on Pro, all assists off except damage, this is something that has bothered me too. My POV is neutral. Almost like luck, sigh in relief when it’s easy, get burned when it’s not. Never played previous Forzas except 1 so I don’t know what’s the norm and I found 1 hard, never finished it.
Most singleplayer races I run for the career mode are (thankfully) manageable provided you don’t make too many panic mistakes or brake too late, but usually at the lower classes and on-road. I don’t dare play S1/S2/X999 because of the control - it don’t matter if muscle, tuner, exotic. Anything with big power and no TC/STM will wheelspin to hell and back. But that’s physics so I can accept that.
Think 2 or 3 races in Surfer’s Paradise had me restarting god knows how many times yet I could never get in front. Is the ‘rubberband detection’ on any Horizon Edition car based on its default HE spec with big power and high speed? When I downgraded a HE Viper back to stock engine and most stock parts, a Chevy SS !!4-door!! of all things would blast past me while I’m in a draft about to slingshot and I’d never catch up. But then it’s still ?S1? class, even a standard SRT-10 can’t go down to A or less, and being in the same class with half the power is virtual suicide. Yet if you reset it to default HE upgrades, you can’t handle it in a tight circuit while AI runs effortlessly past you.
I gave up and came dead-last twice accepting the results to get some credits/XP at least. I tried again later (forgot if same or different car, maybe a standard non-bonus Viper that was stock?) and suddenly I was (at least fighting it out, not sailing) in front. Almost as if the game demanded that I must “lose a few times no matter what, even if it’s not a multiplayer race”.
At least those were cars on asphalt. I know there was one particular SUV trail race also in that area, you jump into the lake twice or thrice I think, I went in with a stock Urus (so now the rubberband is not supposed to go too crazy) but never made it to first. Forgot if I restarted enough to get just one run to first, or if I made do with a lower placing (or gave up completely), and continued with career mode racing other events. I admit repeated severe mistakes on throttle and racing line, but I blame that on my own frustration due to the restarts (and my controller failing to detect my turning inputs, because of me pressing the buttons harder in frustration).
The Midnight Battle for the M3 was another one that made me really fed up too. I got lucky with that one, didn’t restart too much (felt like 10 to 20), but I never would have won it if I used some big-turbo street car that can’t launch or turn or used something lower (e.g. S1 or A class). Haven’t done too many street races.
I do see AI cars screw up, but for me it’s rare. Sometimes they let me pass, sometimes not. And when the game prompts me to bump it up to Unbeatable? Please NO! I’m not going to torture myself when people play games to relax and have fun! FM6A Yas Marina with the Atoms was hell to me as it is. 
I can still manage for now, just several races in valley/outback area I can’t do because of the game hanging in these places.