Camera movements around a sphere(planet)

Hello(again ^^)!
 
For a few days now I have been working on my camera movement around my sphere, I've been googling and reading guides all over the internet. The plan right now is to have the user go north/south and east/west with the keyboard, strafing in different directions while keeping the distance from origin. I then want the user to be able to turn the camera by pressing the right mousebutton and moving the camera around. The user will also be able to zoom by using the mouse wheel, but right now I have but that on the keyboard as well. So that is the plan, atleast for now.

I will post the camera code here before continueing:

XMFLOAT3 up, position, lookAt;
XMVECTOR upVector, positionVector, lookAtVector;
float yaw, pitch, roll, radians;
XMMATRIX rotationMatrix;
 
up.x = 0.0f;
up.y = 1.0f;
up.z = 0.0f;
 
upVector = XMLoadFloat3(&up);
 
position.x = mPositionX;
position.y = mPositionY;
position.z = mPositionZ;
 
positionVector = XMLoadFloat3(&position);
 
lookAt.x = 0.0f;
lookAt.y = 0.0f;
lookAt.z = 1.0f;
 
lookAtVector = XMLoadFloat3(&lookAt);
 
pitch = mRotationX * 0.0174532925f;
yaw = mRotationY * 0.0174532925f;
roll = mRotationZ * 0.0174532925f;
 
rotationMatrix = XMMatrixRotationRollPitchYaw(pitch, yaw, roll);
 
lookAtVector = XMVector3TransformCoord(lookAtVector, rotationMatrix);
upVector = XMVector3TransformCoord(upVector, rotationMatrix);
 
lookAtVector = XMVectorAdd(positionVector, lookAtVector);
 
mViewMatrix = XMMatrixLookAtLH(positionVector, lookAtVector, upVector);
 
return;

I first started by thinking around the rotationX, Y and Z and got that working as long as I didn't lookAtVector = XMVectorAdd(positionVector, lookAtVector);(is this what is called translating?) but since I wanted to give the user the ability to turn around as if he/she was using a first person camera I dropped that idea. I still believed if I would have gotten this to work it would have been the best solution.

I then started working with the positionX, Y and Z and got that working pretty nicely or well closer then the rotation atleast. The issue I have currently having with this one is that I have hard time finding a equation that fits all the cases. I have no issue working with the XZ-plane only(East/West) but since the x also is included in the XY plane(North/South) I'm only able to get it to work in one of the directions by having an angleXZ and angleXY which I increase or decrease depending on which button is pressen. is there a general equation that covers all the directions? Or is the rotation solution I should go for?

I find working with the position vector the easiest to understand but I think that the rotation is the way to go but my lack of knowledge in the Matrix area prohibits me to continue down that path.

Also as before my code have the framework used in the Rastertek guide's which have one Camera class and one Position class.

Best Regards and Thanks in Advance!

//Toastmastern