Many bengals, especially young ones, aren't really cuddlers. They just aren't, and may never be, so you can't hold that expectation against them. For mine, one of mine now regularly cuddles with me, but it took her 5 years to figure out that this is what she wanted to do
The other one still isn't. Neither are fearful of me though, and we have a great relationship. I'm just throwing this out there as an example, so that you don't judge your bengals by what another cat was like.
As far as your cat liking you in particular, I would suggest thinking of it this way... every time he comes into contact with you, his attitude towards you shifts a small amount, based on the encounter. He walks past you in the hallway and you step on his tail, he becomes a little more fearful. He walks up to you in the kitchen and you give him a nice fat piece of chicken, and his estimation of you goes up a little bit, etc. You mentioned taking him to the vet, that's obviously a big negative for him. But playing with bengals is a really really big deal, so if you are his play buddy then that is going to average out in a hurry. So what you have to kind of do, is to be aware of how each interaction with him looks from his perspective, and try to tip the balance in your favor.
If one person in the family is the person who takes him to the vet, and administer pills, and give baths, and administer flea treatments etc., and another is the one that plays with him and gives him snacks etc., then who is he going to like better? So you just need to think about that, and try to tip things in your favor. Perhaps distribute the "bad stuff" amongst other family members if possible. More play, more treats, less trying to pet him or pick him up (if he's not receptive to that) and so forth for you. He definitely will come around, if you give him a reason to.