Pitfalls and Errors When Using AVAssetWriter to Save the OpenGL Backbuffer Into a Video.

For my current game I need to record the OpenGL back buffer of GLKView into a video using AVAssetWriter.

I will not go over all the code to do this(which is not that much).

However, I will go quickly over some of the pitfalls that might make you go scratching your head when you try to create a video on iOS.

The first thing to consider is that AVAssetWriter might fail without throwing an exception and without returning any value pointing that out.

To test if there was an error within AVAssetWriter methods you need to read the status and error properties of AVAssetWriter.

You can print the status and error like so:

NSLog(@"%ld %@", (long)videoWriter.status, videoWriter.error);

The first pitfall I encountered is that after startWritting my AVAssetWriter status change to AVAssetWriterStatusFailed.

(Or the number 3)

The reason I was getting this error is because I didn’t delete the file I tried to write into from the previous run session.

That was an easy error but after that I was getting an error when trying to append the first(and following) CVPixelBuffer.

I got AVAssetWriterStatusFailed again but the error value was Unknown Error or the OSStatus was -12902.

After a while I have found out that nothing was wrong with my code except for the fact that something was wrong with the resolution.

Since I was running my app on an iPad3 the resolution was 2048×1536. I don’t know why but the codec or AVAssetWriter couldn’t handle this resolution.

Simply reducing the resolution to something smaller allowed me to write the video without getting any error.

 

For the sake of completion here is part of the code I was using:

 

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
    RenderGame();
    @synchronized(self) {
        if (Recording)
        {
 //           sample = [self sampleBufferFromCGImage:[self snapshotRenderBuffer]];
            GLubyte * data = [self snapshotRenderBufferToData];
            CVPixelBufferRef pixelBuffer = [self pixelBufferFromBytes: &data withSize:CGSizeMake(videoWidth, videoHeight) withSourceSize: CGSizeMake(screenWidth, screenHeight)];
            if (adaptor.assetWriterInput.readyForMoreMediaData)
            {
                RecordFrame++;
                [adaptor appendPixelBuffer:pixelBuffer withPresentationTime:CMTimeMake(RecordFrame*10, 600)];
            }
            //                [writerInput appendSampleBuffer: sample];
            CFRelease(pixelBuffer);
            free (data);
            NSLog(@"%ld %@", (long)videoWriter.status, videoWriter.error);
            //            NSLog(@"%ld", (long)videoWriter.status);
        }
    }
}
-(IBAction)selectRecord:(id)sender
{
    if (DoneRecord)
        return;
    @synchronized(self) {
        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        NSString *documentsDirectory = [paths objectAtIndex:0];
        NSString *sourcePath = [documentsDirectory stringByAppendingPathComponent:@"video.mov"];
        if (Recording)
        {
            [writerInput markAsFinished];
            NSLog(@"%ld", (long)videoWriter.status);
            [videoWriter finishWritingWithCompletionHandler:^{
                UISaveVideoAtPathToSavedPhotosAlbum(sourcePath,nil,nil,nil);
            }];
            DoneRecord = YES;
        }
        else
        {
            videoWidth = screenWidth/2;
            videoHeight = screenHeight/2;
            NSError * error = nil;
            NSFileManager *fileManager = [NSFileManager defaultManager];
            BOOL success = [fileManager removeItemAtPath:sourcePath error:&error];
            if (success) {

            }
            else
            {
                NSLog(@"Could not delete file -:%@ ",[error localizedDescription]);
            }
            RecordFrame = 0;
            videoWriter = [[AVAssetWriter alloc] initWithURL:
                                          [NSURL fileURLWithPath:sourcePath] fileType:AVFileTypeQuickTimeMovie
                                                                      error:&error];
            NSLog(@"%ld", (long)videoWriter.status);
            NSParameterAssert(videoWriter);

            NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                           AVVideoCodecH264, AVVideoCodecKey,
                                           [NSNumber numberWithInt:videoWidth], AVVideoWidthKey,
                                           [NSNumber numberWithInt:videoHeight], AVVideoHeightKey,
                                           nil];
            writerInput = [AVAssetWriterInput
                                                assetWriterInputWithMediaType:AVMediaTypeVideo
                                                outputSettings:videoSettings]; //retain should be removed if ARC

            NSParameterAssert(writerInput);
            NSParameterAssert([videoWriter canAddInput:writerInput]);
            writerInput.expectsMediaDataInRealTime = YES;
            [videoWriter addInput:writerInput];

            adaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput sourcePixelBufferAttributes:nil];

            [videoWriter startWriting];
//            NSLog(@"%ld %@", (long)videoWriter.status, videoWriter.error);
            [videoWriter startSessionAtSourceTime:kCMTimeZero];
        }
        Recording = !Recording;
    }
}

Racing Car with Springs Custom Physics Simulation (Part 3)

Preface

In Part 2 we made a basic physics simulation of a car.

The car model was made of 5 points(4 wheels and the car center).

We included gravity in the simulation and we were able to drive over a terrain and fall from ledges.

In this part we will add suspensions and the car will be able to jump over ramps and drive smoothly over small bumps.

The Models

The model of the track remains the same as in part 1 and 2(a set of 3D triangles).

The car is made out of 5 points(4 for the wheels and 1 for the center) like in Part 2.

We will also save the gravity velocity of each of these 5 points from the previous frame and also save the car’s orientation from the previous frame.

Unlike part 2 we will also add suspensions to the wheels which will be modeled with springs.

The resting position of the wheels will be lower than the car’s center(a lower wheel base).

Our wheel base will be 1.8 m wide, 4 m long and 1 m (which is also the spring length) bellow the car’s center.

Steering

Like in part 2 we calculate the new Look and Right vectors(the orientation) of the car by rotating the Look and Right vectors from the previous frame on the Look X Right plane.

However, this time we will only do so if the front wheels were touching the terrain in the previous frame.

 

double DeltaAngle = WheelFactor*t*AngleSpeed;
if (!LastFrontWheelsTouch)
	DeltaAngle = 0.0;
Look = LastRight*sin(DeltaAngle)+ LastLook*cos(DeltaAngle);
Right = LastRight*cos(DeltaAngle)- LastLook*sin(DeltaAngle);

 

The Suspensions(Springs)

In the real world, if cars didn’t have suspensions every small bump in the road would rattle the car. The suspensions allow the car to drive smoothly over uneven road and obstacles.

In part 2 whenever the car drove over a small bump it would move quite violently, or even worse, it would make the car jump.

With the steering we use now, which only function if the front wheels touch the surface, these small bumps would make the steering very difficult.

We model the car’s suspensions using a spring equation. One spring for each one of the 4 wheels.

The basic ODE(Ordinary Differential Equation) of a spring is:

m*x” = -k*x

An ODE is a certain way to express one or several equations implicitly(if they even exist).

In the ODE above is the car’s mass, x” is the acceleration, k is the spring’s strength constant and x is the distance of the edge of the spring(or the wheel attached to the spring) from it’s resting position.

A spring is a device that apply force whenever it is not in it’s resting form. Like when it’s stretched or compressed.

m*x” is force expressed as mass multiplied by acceleration.

So we can see that the force the spring applies is the opposite direction to the distance of the spring’s free edge from it’s resting position(multiplied by a factor k).

How does this equation help us?

We could try to apply the force to our wheels every frame but that won’t be accurate.

Since this equation is non linear calculating it in the frame’s delta time granularity will give us bad results.

What we actually want is to analytically extract the motion equation from the ODE.

The movement of the spring depending on the time t and with the initial conditions c1 and c2 is:

x(t) = c2*sin(sqrt(k/mass)*t)+c1*cos(sqrt(k/mass)*t)

We can derive this equation and get a similar equation for the velocity of the spring:

v(t) = sqrt(k/mass)*(c2*cos(sqrt(k/mass)*t)-c1*sin(sqrt(k/mass)*t))

Our initial conditions c1 and c2 are found at time t=0.

c1 = x(0)

x2 = v(0)/sqrt(k/mass)

Ok we got all the initial conditions and we can now assign t to the equations to get the spring’s length and veolocity.

But how does this spring interact with the world?

Well what we actually do is recalculate the initial condition every frame.

The spring’s current length and velocity are adjusted from the interaction with the track and are fed again into the spring equations in the next frame.

Since the equation is agnostic to the initial time we can assume that time starts from t=0 every frame and only use the frame’s time delta as the time we want to calculate the new motion of the spring.

The code for updating the spring motion is the following:

 

for (unsigned int i=0; i<WheelsSpring.size(); i++)
{
	double c1 = WheelsSpring[i];
	double c2 = WheelsSpringSpeed[i]/SqrtSpringMassK;

	WheelsSpringSpeed[i] = (SqrtSpringMassK*c2*cos(SqrtSpringMassK*t)-SqrtSpringMassK*c1*sin(SqrtSpringMassK*t));
	WheelsSpring[i] = (c2*sin(SqrtSpringMassK*t)+c1*cos(SqrtSpringMassK*t));
}

 

Energy Loss

There is a problem with the springs we used above.

They do not lose energy.

It means that if the car’s springs are not in the resting position they will oscillate forever even if the car stays in the same place.

We model springs with energy loss with the following ODE:

m*x” = -k*x -c*x’

The springs force is now affected by the spring’s velocity.

The energy loss constant is and x’ is the spring’s velocity.

Much like in the previous ODE I used an online ODE solver and found the equation for the motion in dependency of the time t.

I then derived the equation to get the velocity equation.

After finding the two equations I assigned t=0 to the two equations to find the initial conditions.

The last step was to calculate an energy loss constant that made sense.

If the energy loss constant is 0 then you will get a square root of a negative number which is an imaginary number.

This actually makes sense because this imaginary value is the power of two exponents.

An imaginary power of an exponent is also an imaginary number but when you add two exponents with symmetric imaginary numbers the imaginary part “disappears” and you get the same equation we had with the previous ODE.

The code for updating the spring motion based on these equations is:

 

for (unsigned int i=0; i<WheelsSpring.size(); i++)
{
	double a1 = (WheelsSpring[i]*SpringParaboleNegative-WheelsSpringSpeed[i])/SpringParabole;
	double a2 = (WheelsSpring[i]*SpringParabolePositive+WheelsSpringSpeed[i])/SpringParabole;

	WheelsSpringSpeed[i] = -a1*SpringParabolePositive*exp(-t*SpringParabolePositive)+a2*SpringParaboleNegative*exp(t*SpringParaboleNegative);
	WheelsSpring[i] = a1*exp(-t*SpringParabolePositive)+a2*exp(t*SpringParaboleNegative);
}

For the sake of completion here is the code I used to initialize the constants:

CarMass = 30.0;
SpringK = 5000.0;
SpringDamp = sqrt(4.0*SpringK*CarMass)*5.;
SpringParabole = sqrt(SpringDamp*SpringDamp-4.0*SpringK*CarMass)/CarMass;
SpringParabolePositive = (SpringParabole/2.0)+SpringDamp/(2.0*CarMass);
SpringParaboleNegative = (SpringParabole/2.0)-SpringDamp/(2.0*CarMass);
SqrtSpringMassK = sqrt(SpringK/CarMass);

 

Simulation

Our simulation consists of several steps.

  1. The first step is the steering(in case the front wheels were on track in the previous frame).
  2. The second step is updating the springs.
  3. The third step is adding the thrust of the car(only if the front wheels were on the track in the previous frame). We also apply gravity and update the car and wheel’s position with the current velocity.
  4. The fourth step is testing whether the wheels penetrate the track and by how much. If they do we adjust the springs length and velocity and also the gravity velocity of the wheels and the center of the car.
  5. The fifth and last step is recalculating the orientation of the now deformed wheel base.

In the third step we basically add the thrust according to the wheel base orientation and we make sure that we don’t add thrust when the car’s velocity is equal or greater to the car’s maximum speed.

Testing for wheel penetration is done similar to what we did in part 2.

We cast a ray from slightly above the wheel’s position down to each triangle in the terrain.

If the ray intersects a triangle we can tell if the wheel penetrates that triangle and by how much according to the where the ray intersect the triangle.

(We presented an optimization to this test in part 2 so that we wouldn’t need to test against all the triangles for each wheel).

This time, as opposed to in part 2, we do not adjust the wheel’s position directly if there is a penetration.

Instead, we adjust the wheel’s spring length and adjust the wheels velocity.

(Notice the wheels velocity and the spring’s speed are kept in separate variables).

If the spring’s speed pushes the spring velocity downwards into the triangle, we zero this speed and we add it to the wheel’s velocity and also to the car’s center velocity.

(This part needs more work though, as the result are not good in all the scenarios).

This is where the springs interact with the terrain and the initial conditions of the spring equations are modified.

The final part is restoring the wheel base from it’s deformed state and calculate the new car orientation.

(Quote from part 2)

We will restore the original form of the wheel base by treating the 4 wheels as if they have springs among themselves(a total of 6 springs).

This will make the 4 wheel points simulate the wheels base as a if it was a rigid body.

In order to restore the original form of the wheels base we go over all the 6 springs and adjust them to be closer to their original length.

For the sake of completion here is the update code:

void Update (double t)
{
	if (Input.GetLeft())
		WheelFactor -= t/AngleLatency;
	else if (Input.GetRight())
		WheelFactor += t/AngleLatency;
	else
		WheelFactor = std::max(std::fabs(WheelFactor)-(2.0*t/AngleLatency), 0.0)*(WheelFactor>0.0?1.0:-1.0);
	WheelFactor = std::max(std::min(WheelFactor, 1.0), -1.0);

	double CurrentFlatSpeed = 0.0;
	if (Input.GetThrust())
		CurrentFlatSpeed = MaxFlatSpeed;

	// Step 1: Steering.
	double DeltaAngle = WheelFactor*t*AngleSpeed;
	if (!LastFrontWheelsTouch)
		DeltaAngle = 0.0;
	Graphics2D::Position Look = LastRight*sin(DeltaAngle)+ LastLook*cos(DeltaAngle);
	Graphics2D::Position Right = LastRight*cos(DeltaAngle)- LastLook*sin(DeltaAngle);
	Graphics2D::Position Up = Look.Cross(Right).Normalize();
	Graphics2D::Position YAxis = Graphics2D::Position (0, 1, 0);

	// Step 2: Update springs.
	for (unsigned int i=0; i<WheelsSpring.size(); i++)
	{
		double a1 = (WheelsSpring[i]*SpringParaboleNegative-WheelsSpringSpeed[i])/SpringParabole;
		double a2 = (WheelsSpring[i]*SpringParabolePositive+WheelsSpringSpeed[i])/SpringParabole;

		WheelsSpringSpeed[i] = -a1*SpringParabolePositive*exp(-t*SpringParabolePositive)+a2*SpringParaboleNegative*exp(t*SpringParaboleNegative);
		WheelsSpring[i] = a1*exp(-t*SpringParabolePositive)+a2*exp(t*SpringParaboleNegative);
	}

	// Step 3: Thrust, gravity and position update.
	if (LastFrontWheelsTouch)
	{
		double ThrustSpeed = CurrentVelocity.Dot(Look);
		if (!Input.GetThrust())
			CurrentVelocity = CurrentVelocity-Look*std::max(std::min(ThrustSpeed, 0.25*MaxFlatSpeed*t/AccelLatency), -0.25*MaxFlatSpeed*t/AccelLatency);
		double ForwardSpeed = std::max(CurrentVelocity.Dot(Look), 0.0);
		double DownSpeed = std::min(CurrentVelocity.Dot(Up), 0.0);
		double RightSpeed = CurrentVelocity.Dot(Right);
		RightSpeed = std::min(fabs(RightSpeed), MaxFlatSpeed*t/AccelLatency)*(RightSpeed>0.0?1.0:-1.0);
		CurrentVelocity = CurrentVelocity+Look*(std::min(std::max(MaxFlatSpeed-ForwardSpeed, 0.0), CurrentFlatSpeed*t/AccelLatency))-Right*RightSpeed-Up*(LastCenterTouch?DownSpeed:0.0);
	}
	CurrentVelocity=CurrentVelocity+Gravity*t;
	for (unsigned int i=0; i<WheelsGravityVelocity.size(); i++)
		WheelsGravityVelocity[i]=WheelsGravityVelocity[i]+Gravity*t+(LastCenterTouch?YAxis*Look.Dot(YAxis)*CurrentFlatSpeed*t:Graphics2D::Position(0, 0, 0));
	CarParms->SetLook (Look, Graphics2D::Position(0, 1, 0));
	std::vector<Graphics2D::Position> Wheels;
	Wheels.resize(WheelsDelta.size());
	for (unsigned int i=0; i<Wheels.size(); i++)
		Wheels[i] = Right*WheelsDelta[i].x+Look*WheelsDelta[i].z+Up*WheelsDelta[i].y+Pos+(Look*CurrentFlatSpeed+WheelsGravityVelocity[i])*t;

	LastSteerTouch = false;
	LastCenterTouch = false;

	Pos = Pos+CurrentVelocity*t;
	double TouchDistance = 1.*(std::max(-WheelsSpring[0], std::max(-WheelsSpring[1], std::max(-WheelsSpring[2], std::max(-WheelsSpring[3], 0.0))))+SpringLength);
	LastFrontWheelsTouch = false;
	Pos.y = std::max(0.0, Pos.y);

	std::list<unsigned int>::iterator q;
	CarParms->SetPosition (Pos);

	// Step 4: Penetration test and handling
	std::vector<bool> isTouch;
	isTouch.resize(4);
	std::list<Graphics2D::Position> Normals;
	Graphics2D::Position PreviousVelocity = CurrentVelocity;
	for (unsigned int i=0; i<Wheels.size(); i++)
	{
		if (Up.y<0.00001)
			continue;
		Graphics2D::Position p = Wheels[i]+Up*SpringLength;
		unsigned int StartX = std::min((unsigned int)(std::max((p.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
		unsigned int StartZ = std::min((unsigned int)(std::max((p.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);
		p = Wheels[i];
		Graphics2D::Position p2 = p-Up*SpringLength;
		unsigned int EndX = std::min((unsigned int)(std::max((p2.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
		unsigned int EndZ = std::min((unsigned int)(std::max((p2.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);
		if (EndX<StartX)
		{
			unsigned int KeepX = StartX;
			StartX = EndX;
			EndX = KeepX;
		}
		if (EndZ<StartZ)
		{
			unsigned int KeepZ = StartZ;
			StartZ = EndZ;
			EndZ = KeepZ;
		}

		double ElasticEnergyLoss = 0.9;
		bool WheelTouch = false;
		for (unsigned int CountZ = StartZ; CountZ<=EndZ; CountZ++)
			for (unsigned int CountX = StartX; CountX<=EndX; CountX++)
			{
				std::list<unsigned int>::iterator q;
				for (q = TrackGrid[CountZ][CountX].begin(); q != TrackGrid[CountZ][CountX].end(); q++)
				{
					const math::Ray r(float3(p.x, p.y, p.z)+float3(Up.x, Up.y, Up.z)*10.0*SpringLength, -float3(Up.x, Up.y, Up.z));
					float d = 0;
					math::float3 Point;
					if (TrackGeometry[*q].Intersects(r, &d, &Point))
					{
						double UpLength = r.pos.y/Up.y;
						double UpWheelHeight = Wheels[i].y/Up.y;
						if (UpLength-d>=UpWheelHeight)
						{
							double Penetrate = std::max(UpLength-d, 0.0)-UpWheelHeight;
							if (Penetrate>WheelsSpring[i])
							{
								WheelsSpring[i] = std::max(WheelsSpring[i], Penetrate);//std::min(Penetrate, SpringLength));
								double DownSpringSpeed = -std::min(WheelsSpringSpeed[i], 0.0);
								CurrentVelocity = CurrentVelocity+Up*std::max(DownSpringSpeed+std::min(WheelsGravityVelocity[i].y, 0.0), 0.0)/(double)WheelsGravityVelocity.size();//*EnergyLossFactor;
								if (Penetrate>SpringLength)
									WheelsGravityVelocity[i].y = std::max(0., WheelsGravityVelocity[i].y);//-DownSpringSpeed;
								else if (WheelsGravityVelocity[i].y<0.0)
								{
									WheelsGravityVelocity[i].y+=DownSpringSpeed*Up.y;
									WheelsGravityVelocity[i].y = std::min(WheelsGravityVelocity[i].y, 0.0);
								}
							}
							if (i<2)
								LastFrontWheelsTouch = true;
							WheelTouch = true;
							isTouch[i] = true;
						}
					}
				}
			}
		if (WheelTouch)
			continue;
		if (FloorHeight>=Wheels[i].y)
		{
			double Penetrate = FloorHeight-Wheels[i].y/Up.y;
			if (Penetrate>WheelsSpring[i])
			{
				WheelsSpring[i] = std::max(WheelsSpring[i], Penetrate);//std::min(Penetrate, SpringLength));
				double DownSpringSpeed = -std::min(WheelsSpringSpeed[i], 0.0);
				CurrentVelocity = CurrentVelocity+Up*std::max(DownSpringSpeed+std::min(WheelsGravityVelocity[i].y, 0.0), 0.0)/(double)WheelsGravityVelocity.size();//*EnergyLossFactor;
				if (Penetrate>SpringLength)
					WheelsGravityVelocity[i].y = std::max(0., WheelsGravityVelocity[i].y);//-DownSpringSpeed;
				if (WheelsGravityVelocity[i].y<0.0)
				{
					WheelsGravityVelocity[i].y+=DownSpringSpeed*Up.y;
					WheelsGravityVelocity[i].y = std::min(WheelsGravityVelocity[i].y, 0.0);
				}
			}
			if (i<2)
				LastFrontWheelsTouch = true;
			isTouch[i] = true;
		}
		else
		{
		}
	}

	// Step 5: Wheel base correction and orientation calculation
	for (unsigned int k=0; k<10; k++)
	{
		for (unsigned int i=0; i<Wheels.size(); i++)
			for (unsigned int j=i+1; j<Wheels.size(); j++)
			{
				Graphics2D::Position v = Wheels[i]-Wheels[j];
				Graphics2D::Position center = (Wheels[i]+Wheels[j])*0.5;
				double l = (WheelsDelta[i]-WheelsDelta[j]).Length();
				double radius = (0.1*l+0.9*v.Length())/2.0;
				Wheels[i] = (Wheels[i]-center).Normalize()*radius+center;
				Wheels[j] = (Wheels[j]-center).Normalize()*radius+center;
			}
	}
	Look = (Wheels[0]-Wheels[3]).Normalize();
	Look.y = std::min(fabs(Look.y), 0.65)*(Look.y>0.0?1.0:-1.0);
	Look = Look.Normalize();
	Right = (Wheels[1]-Wheels[0]).Normalize();
	Right.y = std::min(fabs(Right.y), 0.65)*(Right.y>0.0?1.0:-1.0);
	Right = Right.Normalize();
	LastLook = Look;
	LastRight = Right;
	CarParms->SetLook(Look, Look.Cross(Right));
	for (unsigned int i=0; i<Wheels.size(); i++)
	{
		TireParm[i]->SetPosition(Right*WheelsDelta[i].x+Look*WheelsDelta[i].z+Look.Cross(Right)*(WheelsSpring[i]+WheelsDelta[i].y)+Pos);
		TireParm[i]->SetLook(Look, Look.Cross(Right));
	}

}

 

Conclusion

In this part we have improved our car model by adding suspensions to the wheels.

The suspensions are modeled with springs.

We used analytic equations derived from ODEs to update the springs’ motion.

The springs were affected by the terrain by adjusting the initial conditions of the springs equations.

This simulation is more realistic but is still lacking in certain scenarios.

Notice we are now testing wheel/triangle intersection with rays that do not necessarily align to the y axis but rather align to the car’s Up vector.

We have also forced the car’s orientation to a certain cone so the car won’t be able to flip upside down. This pose all sort of issues.

In the next part we will try to improve on this.

On the mean time you can see the results of this simulation in the following video:

Racing Game Custom Physics Simulation (Part 2)

Preface

In part 1 we have shown how to make a car(truck) glide on a track made out of 3D triangles.

We were able to steer the car and it was “glued” to the track to make it seem like there are physics involved.

In this part we are going to have an actual physics simulation(to some degree).

The Models

The model of the track remains the same as in part 1(a set of 3D triangles).

The car model is made of 5 points: 4 for the wheels and 1 for the bottom center. Just like in part 1.

However, we will also maintain variables to keep the orientation of the car and the gravity velocity of each of the 4 wheels(in addition to the car’s gravity velocity).

Steering

Since we now maintain the orientation of the car from the previous frame we can’t just assume the steering vector is calculated aligned to the xz plane.

To calculate the new steering vector we rotate the Look and Right vectors by a delta of an angle on the Look x Right plane:

				Look = LastRight*sin(DeltaAngle)+ LastLook*cos(DeltaAngle);
				Right = LastRight*cos(DeltaAngle)- LastLook*sin(DeltaAngle);

The delta of the angle is the angular speed multiplied by the frame’s time delta.

The forward velocity is calculated the same as in part 1 but notice the Look vector might not be on the xz plane this time.

Gravity

This time we are also going to take gravity into account.

Our gravity is the following vector (0, -9.8, 0).

It is a vector pointing downwards with an acceleration of 9.8 meters per square second.

We compose the car’s velocity from two components. The steering velocity from above and the Gravity velocity.

The reason we separate the two is because it would be easier to set the gravity velocity to 0 whenever a point in the model hits the ground or track.

We also need to maintain a separate gravity velocity for each of the 5 points in the car model to make the physics simulation of the car work as if we had an object with volume.

The Simulation

After we calculated the Look and Right orientation vectors and the steering velocity vector we need to apply gravity.

For each of the 5 gravity velocity vectors we add the Gravity vector multiplied by the frame’s time delta.

				CurrentGravityVelocity=CurrentGravityVelocity+Gravity*t;
				for (unsigned int i=0; i<WheelsGravityVelocity.size(); i++)
					WheelsGravityVelocity[i]=WheelsGravityVelocity[i]+Gravity*t;

The next step is to calculate the current position of the wheels and the position of the car center.

For the car center position, we add the current steering velocity vector and the current gravity velocity vector multiplied by the frame’s time delta.

Like so:

				Pos = Pos+(Look*CurrentFlatSpeed+CurrentGravityVelocity)*t;

For each of the 4 wheels we calculate the wheel’s position relative to the car’s center using the wheels base dimensions and the Look and Right orientation vectors.

We then add the car’s center position to each of the 4 wheels positions.

The last step is to add the steering velocity and the gravity velocity multiplied by the time delta. The steering velocity is the same as it was for the car’s center but notice the gravity velocity might be unique for each wheel point(We saved them in their own variables).

				for (unsigned int i=0; i<Wheels.size(); i++)
					Wheels[i] = Right*WheelsDelta[i].x+Look*WheelsDelta[i].z+Pos+(Look*CurrentFlatSpeed+WheelsGravityVelocity[i])*t;

In a similar fashion to part 1 we now check which triangles each one of the 5 points of the model intersect with.

The difference is that instead of setting the points to the intersection height of the respective triangles, we only set them to the intersection height in case their own height is lower.

In addition we set to zero the Gravity velocity of each point only if  it’s height was adjusted by a triangle.

Now we have 5 points of the car displaced into new heights. The car’s center point will be used for the new position, but like in part 1, we need to calculate the new orientation from the 4 wheel points.

The current car model simulates a rigid body. However while adjusting the wheels’ heights the car’s wheel base is now deformed.

We will restore the original form of the wheel base by treating the 4 wheels as if they have springs among themselves(a total of 6 springs).

This will make the 4 wheel points simulate the wheels base as a if it was a rigid body.

In order to restore the original form of the wheels base we go over all the 6 springs and adjust them to be closer to their original length.

We iterate over this process for ten times and at the end we would get something closer to the original form.

				for (unsigned int k=0; k<10; k++)
				{
					for (unsigned int i=0; i<Wheels.size(); i++)
						for (unsigned int j=i+1; j<Wheels.size(); j++)
						{
							Graphics2D::Position v = Wheels[i]-Wheels[j];
							Graphics2D::Position center = (Wheels[i]+Wheels[j])*0.5;
							double l = (WheelsDelta[i]-WheelsDelta[j]).Length();
							double radius = (0.1*l+0.9*v.Length())/2.0;
							Wheels[i] = (Wheels[i]-center).Normalize()*radius+center;
							Wheels[j] = (Wheels[j]-center).Normalize()*radius+center;
						}
				}

Now that we have the wheel points placed on the wheels base frame we can calculate the new Look and Right orientation vectors in a similar fashion we did in part 1.

				Look = (Wheels[0]-Wheels[3]).Normalize();
				Right = (Wheels[1]-Wheels[0]).Normalize();

Conclusion

We now have a more physically based simulation that also support falling off from edges.

The result of this simulation can be seen in this video:

For the sake of completion I am adding the entire code for the update function.

 

			void Update (double t)
			{
				double WheelFactor = 0;
				if (Input.GetLeft())
					WheelFactor = -1;
				else if (Input.GetRight())
					WheelFactor = 1;
				if (Input.GetThrust())
					CurrentFlatSpeed += MaxFlatSpeed*t/AccelLatency;
				CurrentFlatSpeed = std::max(std::min(CurrentFlatSpeed, MaxFlatSpeed), 0.0);

				double DeltaAngle = WheelFactor*t;
				Graphics2D::Position Look = LastRight*sin(DeltaAngle)+ LastLook*cos(DeltaAngle);
				Graphics2D::Position Right = LastRight*cos(DeltaAngle)- LastLook*sin(DeltaAngle);
				CurrentGravityVelocity=CurrentGravityVelocity+Gravity*t;
				for (unsigned int i=0; i<WheelsGravityVelocity.size(); i++)
					WheelsGravityVelocity[i]=WheelsGravityVelocity[i]+Gravity*t;
				CurrentVelocity = Look*CurrentFlatSpeed+CurrentGravityVelocity;
				CarParms->SetLook (Look, Graphics2D::Position(0, 1, 0));
				std::vector<Graphics2D::Position> Wheels;
				Wheels.resize(WheelsDelta.size());
				for (unsigned int i=0; i<Wheels.size(); i++)
					Wheels[i] = Right*WheelsDelta[i].x+Look*WheelsDelta[i].z+Pos+(Look*CurrentFlatSpeed+WheelsGravityVelocity[i])*t;

				Pos = Pos+CurrentVelocity*t;
				if (Pos.y<0.0)
					CurrentGravityVelocity.y = std::max(0., CurrentGravityVelocity.y);
				Pos.y = std::max(0.0, Pos.y);

				unsigned int StartX = std::min((unsigned int)(std::max((Pos.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
				unsigned int StartZ = std::min((unsigned int)(std::max((Pos.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);

				std::list<unsigned int>::iterator q;
				for (q = TrackGrid[StartZ][StartX].begin(); q != TrackGrid[StartZ][StartX].end(); q++)
				{
					const math::Ray r(float3(Pos.x, 100.0, Pos.z), float3(0, -1, 0));
					float d = 0;
					math::float3 Point;
					if (TrackGeometry[*q].Intersects(r, &d, &Point))
					{
						if (r.pos.y-d>=Pos.y)
						{
							CurrentGravityVelocity.y = std::max(0., CurrentGravityVelocity.y);
							Pos.y = r.pos.y-d;
						}
					}
				}
				CarParms->SetPosition (Pos);
//				std::vector<bool> IsWheelContact;
//				IsWheelContact.resize(4, false);
				for (unsigned int i=0; i<Wheels.size(); i++)
				{
					Graphics2D::Position p = Wheels[i];
					unsigned int StartX = std::min((unsigned int)(std::max((p.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
					unsigned int StartZ = std::min((unsigned int)(std::max((p.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);

					std::list<unsigned int>::iterator q;
					for (q = TrackGrid[StartZ][StartX].begin(); q != TrackGrid[StartZ][StartX].end(); q++)
					{
						const math::Ray r(float3(p.x, 100.0, p.z), float3(0, -1, 0));
						float d = 0;
						math::float3 Point;
						if (TrackGeometry[*q].Intersects(r, &d, &Point))
						{
							if (r.pos.y-d>=Wheels[i].y)
							{
								WheelsGravityVelocity[i].y = std::max(0., WheelsGravityVelocity[i].y);
//								IsWheelContact[i] = true;
								Wheels[i].y = r.pos.y-d;
							}
						}
					}
					if (FloorHeight>=Wheels[i].y)
					{
						WheelsGravityVelocity[i].y = std::max(0., WheelsGravityVelocity[i].y);
//						IsWheelContact[i] = true;
						Wheels[i].y = FloorHeight;
					}
				}
				for (unsigned int k=0; k<10; k++)
				{
					for (unsigned int i=0; i<Wheels.size(); i++)
						for (unsigned int j=i+1; j<Wheels.size(); j++)
						{
							Graphics2D::Position v = Wheels[i]-Wheels[j];
							Graphics2D::Position center = (Wheels[i]+Wheels[j])*0.5;
							double l = (WheelsDelta[i]-WheelsDelta[j]).Length();
							double radius = (0.1*l+0.9*v.Length())/2.0;
							Wheels[i] = (Wheels[i]-center).Normalize()*radius+center;
							Wheels[j] = (Wheels[j]-center).Normalize()*radius+center;
						}
				}
				Look = (Wheels[0]-Wheels[3]).Normalize();
				Right = (Wheels[1]-Wheels[0]).Normalize();
				LastLook = Look;
				LastRight = Right;
				Graphics2D::Position Up = Look.Cross(Right);
				CarParms->SetLook(Look, Up);
			}

Simple Truck Racing Physics(Part 1)

Preface

I am working on a new 3D racing game.

For this racing game I need a track with mounds, hills and ramps.

I am going to cover my progress in making this racing game’s physics simulation.

The Models

At this point the track is made of a series of 3D triangles.

The 3D triangles might be constructed in a way that they form a road with mounds, turns or slopes but they don’t have to.

At this point the track geometry is used for both rendering and representing the terrain geometry in the simulation.

The track dimensions I used for testing are 110×110 square meters.

We also have the truck which has a 3D model representing it visually.

Inside the simulation the truck is made out of 5 points. The center bottom of the truck and 4 more points representing the wheels.

The truck’s size is a 2x2x5 cubic meters box.

The wheels base is 1.8×4.4 square meters.

Steering

For the steering of the truck I am saving the truck’s absolute direction inside a single angle.

I calculate the Look vector from the angle like this:

				Look = Graphics2D::Position(sin(CarAngle), 0, cos(CarAngle));

When I want the truck to rotate I add an angular speed multiplied by the frame’s time to the angle I mentioned above.

I then recalculate the Look vector every new frame.

In order for the truck to move forward we need to add the movement vector to the truck’s current position.

The truck’s movement vector is calculated like this:

				Move = Look*CurrentFlatSpeed*t;
				Pos = Pos+Move;

We don’t want the truck’s speed to accelerate instantaneously so we add the maximum speed multiplied by the frame’s time step but divided by the latency we want it to take to reach maximum speed.

				if (Input.GetThrust())
					CurrentFlatSpeed += MaxFlatSpeed*t/AccelLatency;
				CurrentFlatSpeed = std::max(std::min(CurrentFlatSpeed, MaxFlatSpeed), 0.0);

Terrain checks

At this point we can drive and steer the truck but we are completely ignoring the track(or terrain).

In order for the truck to “glide” on the terrain we will go over every triangle in our track mesh and test to see if the (x, z) part of the center bottom of the truck is inside the projection of the triangle on the xz plane.

(The center bottom of the truck is actually it’s position).

In order to test that we use a ray to triangle intersection test while the ray is from (truck position X, 1000, truck position Z) to (truck position X, 0, truck position Z).

If the ray intersects the triangle then the truck’s center is inside the projection of the triangle. We can then extract the height of the intersection between the ray and the triangle and use that as the new height(y axis value) of our truck.

(For the ray/triangle intersection we use MathGeoLib by clb).

This will make our truck go over the track’s topology but the truck will remain aligned as if it was on a flat surface.

In order to recalculate the truck’s alignment we do the same test we did with the truck’s center but with the 4 wheels instead.

Before we do that we calculate the absolute position of the 4 truck wheels from the truck’s wheels base rotated by the truck’s steering angle and added to the truck’s center bottom. Like so:

 

				Look = Graphics2D::Position(sin(CarAngle), 0, cos(CarAngle));
				Right = Graphics2D::Position(0, 1, 0).Cross(Look);
				for (unsigned int i=0; i<4; i++)
					WheelPos[i] = Right*WheelBase[i].x+Look*WheelBase[i].z+Pos;

We now do the same calculation over all the triangles and calculate the new height for each of the 4 wheels.

We then calculate the new Look and Right vectors of the truck from two vectors.

The Look vector will be the vector pointing from the rear left wheel to the front left wheel and the Right vector will be the vector pointing from the front left wheel to the front right wheel.

Don’t forget we want the normalized vectors.

				Look = (WheelPos[0]-WheelPos[3]).Normalize();
				Right = (WheelPos[1]-WheelPos[0]).Normalize();

That’s it. This will give us the following simulation result.

Optimizations

You probably noticed that we went through all the triangles in the track for each of the 5 points in the truck model.

This might be problematic to the performance and most of the triangles won’t intersect with the truck model.

In order to optimize this we prepare a 2D array where each array cell contains a linked list.

The 2D array represents a grid on the xz plane. The grid divides the plane into squares.

Each cell of the 2D array contains a list of all the triangles that their xz plane Axis Aligned Bounding Square intersects with the square in the grid that the cell represents.

This way every square in the grid will have a list that will contain all the triangles that intersect with the square(and maybe a little bit more that don’t).

So every time we want to test a point in the truck model against the track’s triangles we only need to test it against the triangles in the list of the square the point is at.

For the sake of completion here is the code to calculate a 10 by 10 triangle test optimization grid:

 

				std::vector<math::Triangle> TrackGeometry;
				std::vector<std::vector<std::list<unsigned int> > > TrackGrid;

				std::vector<Graphics2D::Position> & Positions = TrackMesh->GetPosition(0);
				std::vector<unsigned int> & Indices = TrackMesh->GetIndex(0);
				TrackGrid.resize(10);
				for (unsigned int i=0; i<TrackGrid.size(); i++)
					TrackGrid[i].resize(10);
				TrackGeometry.resize (Indices.size()/3);
				Min = Positions[0];
				Max = Positions[0];
				for (unsigned int i=0; i<Positions.size(); i++)
				{
					Min.x = std::min(Min.x, Positions[i].x);
					Min.y = std::min(Min.y, Positions[i].y);
					Min.z = std::min(Min.z, Positions[i].z);
					Max.x = std::max(Max.x, Positions[i].x);
					Max.y = std::max(Max.y, Positions[i].y);
					Max.z = std::max(Max.z, Positions[i].z);
				}
				for (unsigned int i=0; i<TrackGeometry.size(); i++)
				{
					Graphics2D::Position LocalMin, LocalMax;
					LocalMin = Positions[Indices[i*3]];
					LocalMax = Positions[Indices[i*3]];
					for (unsigned int k=1; k<3; k++)
					{
						LocalMin.x = std::min(LocalMin.x, Positions[Indices[i*3+k]].x);
						LocalMin.z = std::min(LocalMin.z, Positions[Indices[i*3+k]].z);
						LocalMax.x = std::max(LocalMax.x, Positions[Indices[i*3+k]].x);
						LocalMax.z = std::max(LocalMax.z, Positions[Indices[i*3+k]].z);
					}
					TrackGeometry[i].a = float3(Positions[Indices[i*3]].x, Positions[Indices[i*3]].y, Positions[Indices[i*3]].z);
					TrackGeometry[i].b = float3(Positions[Indices[i*3+1]].x, Positions[Indices[i*3+1]].y, Positions[Indices[i*3+1]].z);
					TrackGeometry[i].c = float3(Positions[Indices[i*3+2]].x, Positions[Indices[i*3+2]].y, Positions[Indices[i*3+2]].z);
					unsigned int StartX = std::min((unsigned int)(std::max((LocalMin.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
					unsigned int StartZ = std::min((unsigned int)(std::max((LocalMin.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);
					unsigned int EndX = std::min((unsigned int)(std::max((LocalMax.x-Min.x)/(Max.x-Min.x), 0.0)), TrackGrid[0].size()-1);
					unsigned int EndZ = std::min((unsigned int)(std::max((LocalMax.z-Min.z)/(Max.z-Min.z), 0.0)), TrackGrid.size()-1);
					for (unsigned int z1=StartZ; z1<=EndZ; z1++)
						for (unsigned int x1=StartX; x1<=EndX; x1++)
							TrackGrid[z1][x1].push_front(i);
				}
				WheelsDelta.resize(4);
				WheelsDelta[0] = Graphics2D::Position(-0.9, 0, 2.2);
				WheelsDelta[1] = Graphics2D::Position(0.9, 0, 2.2);
				WheelsDelta[2] = Graphics2D::Position(0.9, 0, -2.2);
				WheelsDelta[3] = Graphics2D::Position(-0.9, 0, -2.2);

 

What’s next?

Our current simulation doesn’t have much of a physics feel to it.

The car is basically glued to the terrain. We also don’t deal with ledges.

In part 2 the simulation will get more interesting.

Afternoon Hero is available on Google Play and Amazon App store!


I released a new mobile game! Yay!

It’s available on Google Play and the Amazon App store.

GooglePromo

 

en_generic_rgb_wo_60

6a0148c71fb71b970c014e8a07bf5a970d-800wi

 

 

Afternoon hero is a short, sweet and very old school like platformer for Android.

I think it is actually fun to play and a lot of people seem to think the same.

I am selling it for about $3.

Sadly there doesn’t seem to be a way to give redeem codes for the app on Google Play…

Although  remember you can buy the app and then get a refund if no longer than 15 minutes have passed since the purchase.

It might not be enough to get a full impression but it’s better than nothing.

Here is the Promo video which might give you an idea what is the game about:

 


Screenshot:

ScreenshotJump

Recognize ending contact with the walls and the floor for a platformer game using Box2D

I am working on a new 2D platformer game.

I have been using Box2D for the physics.

For the level itself I used a single b2ChainShape which includes both the floor and the walls.

For my game I need to recognize when the character contact the wall and when it contacts the floor but more importantly when the character ends the contact with the floor or wall.

I could have used two separate fixtures(one for the walls and one for the floor) but according to Box2D I wouldn’t get the perfect collision detection as I would with a single shape.

By implementing b2ContactListener you can listen to when two bodies have a  new contact point and when they end the contact.

To recognize if the character contacts either the wall or the floor I used GetWorldManifold on b2Contact which is provided as a parameter of BeginContact.

b2WorldManifold contains the normal of the contact surface. With the normal I can easily recognize if the contact point is with a wall or the floor.

However,  on EndContact you cannot get b2WorldManifold or the data you will get is garbage.

So how can we tell when we end the contact with the floor rather than the wall?

The solution is to keep 3 counters: The total contact points, the left walls contact points and the right wall contact points.

The total contact points counter includes all the walls and the floor contact points.

When we want to know if we are in contact with the walls and not the floor(like touching the wall mid air) we simply subtract the wall contact counters from the total contacts counter.

When the EndContact method is called we reduce the total contacts counter by one and zero out the walls contact counters.

This will work if the walls are perpendicular to the floor(like a  tile based game).

A more complex level might need a better solution.

 

Platformer screenshot

Simple yet important tip for passing std::string by reference.

Using C++ I quickly learned that I should pass large std::vector by reference.

Passing a std::vector by reference (using &) as a parameter for a function will save you from the operation of copying the std::vector into a local copy.

For my next game I was getting bad performance parsing jsons.

It turns out that nearly the entire CPU time spent on parsing the json was due to passing std::string by value in internal functions.

Passing std::string by reference will save you from a copy operation of that string.