Page 1 of 1

Remade: The Apprentice for PC

Posted: Fri Mar 23, 2018 12:28 pm
by Shikotei
Throughout the years I've had a pet project going on a slow pace. Sometime during 2010-2011 I started working on it.
As with most hobby projects, they either take a long time to get anywhere, or simply never see completion.

I've hinted at having most of the CLUT images, but wanting to get every sprite too here (2012).
And here (2016) once again.
I've shown extensive knowledge on the composition of the level data here (2017).
And (for the first time) stated my goal (2017) here.

As you can see, there've been gaps of several years in which I did nothing or very little.

Early 2017 I got a major breakthrough when the core engine finally became stable enough (and bug-free) that I could actually continue.
The core engine is the part where the level data is read and processed to a list of platforms (and slopes, walls, ceilings) and where Marvin (then just a white square with a black outline) can walk along and jump on.
I've rebuilt this particular section several times before I got it to work the way I needed it to.
Weeks later an unfortunate event happened and I couldn't do much the next half a year.
In august I continued once again (hence the audio decoding thread). Over the course of half a year (then until now) I've been adding, expanding, debugging (a LOT of it) lots of functionality.

Just in case it wasn't clear: me making this post means I have an operational build ready to show the world.
It's not the complete game, but the framework is there.
The game has no menu and jumps straight into the loading screen of The Medieval Tower, Stage 1.
It can be played up until entering the bonus stage.
There is no 'game over' screen, so you won't have to be too careful with lives.
There's also no audio.
This means that:
- The loading screen (overworld map) is implemented
- Stages 1, 2, and 3 can be fully played
- The treasure screen (after each stage) is implemented (yay bonus points!)
- The Medieval Knight (first boss) is fully implemented

Once you enter the bonus stage (Marvin's dream), you can walk and jump. There's no animation or any functionality beyond basic platforms.
Attacking will result in being stuck.

Regarding copyright, I'm quite certain I'm breaking some rules.. but:
Game: The Apprentice is from 1994 (24 years ago)
Developers: The Vision Factory/SPC/SPC Vision went bankrupt in 2002 (16 years ago)
Publisher: Philips Interactive Media International... might exist? Not much info could be easily found. Have they released anything since 1996?
Console: The CD-i has been discontinued in 1998 (20 years ago)

So.. I'd assume there's no-one thinking of suing me (in which case I'll probably cease and desist) or planning to still make a profit from SPC's The Apprentice.
If it's okay with the mods and admins of the forum, I'd like to post the link to the game and upgrade this thread to a dev's log.

Re: Remade: The Apprentice for PC

Posted: Sun Mar 25, 2018 5:54 pm
by goldenband
Just wanted to say that this is an awesome project! :D I probably won't be playtesting since I'm a Mac user and own the original game anyway, but I love it when the standout titles for systems are ported to modern platform. Something similar happened with Dungeons of Daggorath for the Tandy Color Computer, one of my all-time favorite games, though there they at least had source code.

I look forward to future updates!

Re: Remade: The Apprentice for PC

Posted: Fri Apr 13, 2018 11:01 am
by Shikotei
As of just a few minutes ago, the following video is available:

The recording is from last evening, but I've had to do some post-processing. All that really entails is the concatenation of a second video and the removal of some desktop icons.

I'm planning on playing the original game more now that I've hooked everything up for recording high-def. It'll give me the high quality reference material that's apparently nowhere on the internet. I've already noticed that the jump mechanics are way more 'floaty' than what I made.

For now the plan is to get as close as possible to the original source before moving to the next level (even though I've already started a bit with The Well).

Re: Remade: The Apprentice for PC

Posted: Fri Apr 13, 2018 7:07 pm
by cdifan
Just watched the entire video... really *really* good work! :) :)

We know it here, but perhaps you should add to the YouTube description that everything is based on the original sprites, graphics, sound effects, music and level maps? Still an awesome piece of work, in particular knowing as I do that you did almost no reverse engineering on the game programming code...

It even seems much faster than the original, in particular no loading screens :lol:

Re: Remade: The Apprentice for PC

Posted: Sat May 05, 2018 3:19 pm
by Shikotei
I've posted a new video!

The recording is last week's, but there's been little progress since that point.

The changes that were made after the previous video:
- Adjusted jumping physics and animation timing
- Adjusted walking speed
- Adjusted monster animation timing and movement speed
- Adjusted various item animation timing
- Changed behavior for off-screen moving platforms (no longer resets)

As I said last time, I didn't have high quality video to compare my game with, but that's changed now. If you've watched the Let's Play by Seb (HalfBlindGamer) and compared it with the first video, well.. let's just say I wasn't even close :lol:
The new video is much much closer to the real deal. And much better on the eyes too, being less jerky.
If there's one thing I should do ASAP, it's making intros skippable :roll:

The error at the end was introduced when I changed the animation of the flags (in particularly its timing and position). I found the cause some 10-15 minutes later (missing 3rd parameter), so it's fixed now.
There's 3 'flags':
- starting point
- checkpoint
- final flag
I had adjusted the first two (in-game they're the same block of code), but I forgot the final flag, which still used the older method. Their non-compatibility caused a script error.

As for the ongoing issue of legality: I'm working on a 'grey-area' solution.
In simple terms the game will need an image (ISO of some sorts) of the game. One you can easily make of your own copy of the game with Clone CD or perhaps ISObuster.
This is similar to the requirement of the emulator by CDi-fan. It requires a ROM file of an actual CD-i.
The 'copy for private use' should be considered legal (or at least not illegal).

My game would become nothing more than an interface to this file; useless without it!

This file will be opened by the game and all required sprites, sound effects, and music will be extracted.
At this point I have made several attempts at doing this using the same program as the game (MacroMedia Director), but all its functionality for reading large files are.. inadequate.
They either can't open files larger than 500MB (my ISO is 650-ish), are super slow (take 1 minute to write 10MB), or can't even read part of a file...

So.. I'm building it in C#.

I've already built:
- (partial) file reader (keeps RAM usage low)
- table-of-contents builder
- game-file extractor (including a block-reader)
- implemented my M68K emulator (for compiled sprites)
- CLUT-decoder (for basic images and decompiled sprites)
- CDA extractor (for WAV music)
- Compressed ADPCM to ADPCM (for sound effects)

I still need:
- ADPCM decoder
- image combiner (for images that are built up of smaller ones)
- the importer script for the game..

Right now I can extract over 12000 images in 30 seconds. Extracting the full music (some 460MB) takes mere seconds.
I'm still working on the sound effects (they're stored as compressed ADPCM), but I should have that working in the next week.
Most of the code I had already written in different languages, so it's just a matter of translating it.

The challenge that this (extracting all required assets on-the-fly) brings is that any and ALL adjustments to the sprites will also have to be done by the program.
The level tiles in particular are all built up with 3 or 4 smaller tiles. Each normal tile is 20x20 pixels, while the tiles are stored as 4 images of 20x4 pixels...
Many other sprites (monsters, bosses) are split up into smaller images and need to be combined before I can use them in the current setup.

I want the importer script to only extract and import sprites on a need-to-have basis to keep the start up time low.

It sounds like a lot, but on the bright side: it's not hard or anywhere near impossible, it just needs to be done.
I only see challenges.

Re: Remade: The Apprentice for PC

Posted: Wed Sep 26, 2018 6:25 pm
by Shikotei
It may just be a small update in terms of gameplay progress (no progress since last time), but this packs a punch:
It's nearly done!

The file extractor is capable of the following:
- Read part of a file (as small as a single sector)
- Find and build table of contents
- Extract contents of a single file (ie: "level1.dat")
- Build a block-based table of contents
- Completed M68K emulation for purpose of decompiling compiled sprites
- CLUT decoding (basically all images require this)
- CDA extracting (WAV audio)
- Compressed ADPCM decoding to ADPCM (all sound effects are Compressed ADPCM)
- ADPCM decoding to WAV

And now the more complex abilities:
- Resizing of single image (adding empty space to make alignment easy)
- Skipping non-required images (trust me, there's quite the amount of junk files when you decompile sprites without regard to JMP and RTS operations)
- Concatenation of multiple CLUT-stored (and decompiled) images (before they're turned into BMP data)

At this point there's still some challenges left, like the Door sprites.
They're an combination of 3 sprites, of which 2 can be vertically concatenated, but the 3rd has to be overlayed somewhere in the middle.
It's the top half of the door, the bottom half, and the colored lock.
Level1-Doors.png (7.52 KiB) Viewed 5596 times
Sounds easy, right? Not when a program has to do this automatically.

The biggest leap forward came a week or 2 ago, when I actually started building the integration and replaced my processed sprites with the on-demand extracted sprites.
Because the new sprites aren't PNG with transparency, I had to make adjustments to the code that handles the sprites.
They had to get a background color (and background transparent blending). All decompiled sprites have this fluorescent-green color, while most of the CLUT images have pitch-black as background.

Anyways, it works as follows:
- Determine which files need to be extracted (ie: for the first intro: intro1.dat, track3.cda)
- Check if the required sprites are present (if they all are, do not extract)
- Start the extraction process, and regularly check if it's done
- When it's done, import all the sprites
- Mark import as complete and resume the game

The extraction process:
- Create batch file with the required arguments for the file extractor executable
- Open batch file
- Extractor will write a file that marks it is completed
- Delete batch file and marker file

The whole process takes up to a second or two on my system. Depending on the amount of files to be extracted. Some are done in a flash.
I'll probably make a video later today, or tomorrow.

- Have a nearly-completed extractor.
- Can import nearly all sprites.
- It's super fast
- Playable demo is at 98% (arbitrary

Re: Remade: The Apprentice for PC

Posted: Fri Sep 28, 2018 7:53 pm
by Shikotei
Small post, but still important:

Part 1: Principle -
Part 2: Gameplay -

Both are unedited recordings.
Kinda proud of myself right now ^-^

Re: Remade: The Apprentice for PC

Posted: Sat Sep 29, 2018 12:56 am
by cdifan
Shikotei wrote: ā†‘
Fri Sep 28, 2018 7:53 pm
Kinda proud of myself right now ^-^
And you have every right to be! Very good work, and what perseverance...

Re: Remade: The Apprentice for PC

Posted: Thu Oct 25, 2018 3:54 pm
by Shikotei
cdifan wrote: ā†‘
Sat Sep 29, 2018 12:56 am
And you have every right to be! Very good work, and what perseverance...
The trick is to split the huge problem into big ones. And the big into small, and the small ones into even tinier ones. And keep that up until you can solve the issue! Then work your way up into bigger ones that are (by then) partially solved.

And now with even more details!
Some documentation that's way overdue.
Because it's been a while since I looked at the game code, I felt it was necessary to add documentation to some of the complex code. As a refreshing exercise.
Quality varies (a lot), mostly because 90% is just copy-paste of the comments I wrote in. Some of this might not make much sense without the surrounding structure, but you'll have to make do for now.

Extractor documentation is really limited because it's kinda not fully totally completely legal. Most of it is either public domain (BMP and WAV headers, CD-i Greenbook) or so horribly grey-area I won't explain it. The one block of code that is present is also re-written as pseudo-code.

May it help in understanding the structure of both programs:
Documentation on the extractor
Documentation on the game

Re: Remade: The Apprentice for PC

Posted: Fri Feb 01, 2019 6:52 pm
by Shikotei
Let there be an updated video!
Progress as of January 26th 2019

Should you want to show this to others, this video has most (if not all) features as they are now.

There are a few new things to show you!
I've added the introduction video, at last some story!
Now with operational main menu!
And cheats. And start game from any level/stage.
And I've finally added the credits too! They're something special because I couldn't recreate the fade technique without adding (literally) hundreds of images.

All in all this project is still moving forwards, albeit slowly.

The introduction video has an encoded format that I had not yet been able to figure out.
After I found out what it was, decoding it was easy. The format i'm speaking of is an old one, used in the Atari console, called ICE packing.
It's a data cruncher, details can be found in the extractor documentation I mentioned earlier.

One other thing I've changed since September (wow has it been that long since the previous video?) is that now ALL required data is taken from the original game.
I know I said this before, but that was regarding the images and sound. Now the level data (layout) is also directly extracted.

So at the moment, my game requires 3 files to run:
- The file extractor file
- The game file
- The Apprentice IMG file

The first two I can provide (a tiny 4.20MB), the other one you'd have to create using your own physical disc, a disc drive in your PC and some software that can make a proper IMG copy of this physical disc.

Question is.. will I ever dare to release it in the dangerous waters of the internet? There's at least a few people interested.

Re: Remade: The Apprentice for PC

Posted: Mon Mar 11, 2019 9:40 pm
by Shikotei
omegalfa wrote: ā†‘
Tue Feb 05, 2019 12:24 pm
And why not publish it through one of the several retro publishers available? I know some, I can contact them if you want.
Let me know.
As much as I want to release the game, I tend to be very cautiousfor the simple reason that what I created is not purely legal.
The extraction program is definitely not legal.
Iā€™d still have to make adjustments to better prevent abuse of the software. Espesially the extractor; at this point it is too easy to misuse.

How strict are these publishers?
There is one thing I will not supply should there ever be a release: the ISO file of the disc. It is the cornerstone in my attempt to not infringe copyright.

Re: Remade: The Apprentice for PC

Posted: Wed Dec 18, 2019 8:00 pm
by Shikotei
A few months ago I was contacted by a company that claims to have recently acquired the rights to The Apprentice.
We exchanged a bunch of mails where we discussed the possibility to work together to get the game completed.
But negotiations quickly broke once the matter of payment arrived.. it's really sad, but I never heard from them after I named my price. Not even a "too high!" response..

At least the good news is that someone owns the rights and I have their contact information.

There's more big news though; they made me reconsider launching the game as-is (separate extractor and game).
I've deemed the extractor too dangerous to release as a separate executable and have looked for ways to integrate the thing completely into the game.
Unfortunately Director doesn't support this kind of thing (which is the whole reason I had to make a separate program!) so that ruled out a release any time soon on the platform I've made the game so far with.

The past months I've spent finding a way to protect the extractor from abuse and concluded that a different develop environment was the best solution.
Once I decided that I looked into development tools that could do what I wanted to do. My first alternative was Unity. For those unfamiliar with this: a lot of indie developers have used this program to make games for PC and consoles.
After a bunch of tests and proof-of-concept projects, I translated my Lingo code of The Apprentice to C#. This took me roughly two months.

Once this was done I added the extractor (already written in C#) and the integration process was completed.
In short: I could release this version of The Apprentice!

More tests were conducted: I put the game on a USB stick and asked friends if I could borrow their PC to see if the game would run.
Of the ten systems I tried only one gave trouble. Though I ruled out the CPU (Intel and AMD), graphics card, RAM, operating system, and number of monitors as the cause of the issue.
So yes, chances are it won't run on a system, but I am confident there won't be trouble for most of the people out there.

There is one last thing I would like to say:
Please enjoy, and report any bugs you find on my new website!

You can find a link to download the game here.

Re: Remade: The Apprentice for PC

Posted: Thu Dec 19, 2019 12:48 pm
by Bas
AWESOME, Shikotei, that's an amazing job!
Probably the best way to play The Apprentice right now, it goes very smooth! I've contacted you via Facebook and you are on the frontpage of Interactive Dreams, see you there :) ... -cd-i.html

Re: Remade: The Apprentice for PC

Posted: Mon Dec 23, 2019 10:28 am
by Shikotei
Thank you Bas! As I thought it didn't take that long before someone posted a link to an ISO of the disc.. but what can one do?

A new version is now available: 11.3d

It mainly increases support for the various IMG and ISO files that are around. The project page is expanded with tutorials that describe how to create a working image of the disc using ISOBuster and CloneCD.

The other main upgrade is the support for key reassignment! Made possible with an ini file.

It is recommended to update to the latest version.
For the tutorials check the project page.