It is currently 13 Jul 2020, 01:27




 Page 15 of 23 [ 457 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 23  Next
Author Message
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 03 Nov 2017, 04:35 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Very cool - I ordered the FPGA and USB boards tonight, and hopefully I'll be able to build the ADC board eventually once the design is proven...

... edit: looking at things a bit more, I think the DE0-Nano-SOC board with ARM cores, 1GB RAM, and Gigabit Ethernet could actually work better for this - if the ARM cores can actually pull off 300-400Mb/sec. And if not it probably still has enough IO to do the pass through to the FX3.
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 03 Nov 2017, 07:27 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
The DE0-Nano can function at 100Mhz which is the same speed as the FX3's GPIF state-machine - GPIF is used to throw the data over to USB without using the ARM of the FX3 directly. The most time-consuming part of the design so far has been getting reliable and continuous transfer over USB; since the design was 'low production volume' adding in a FPGA allows for a lot of future things - such as performing some of the front-end ld-decode processing directly. There are more powerful FPGA boards out there, but they get expensive fast.

It's a similar story with ADCs; the 10-bit 40MSPS ADC was around 20 bucks a chip. If you want to go faster or wider the price rises exponentially. The good news is that the design is modular; since the FPGA can process data extremely fast, you can change the ADC chip without redesigning the FPGA->FX3->USB3 part. All of the clocks are generated by the FPGA using PLL in software; so you can easily change the ADC speed or the FPGA->FX3 communication speed.

I have a Digilent Nexys Video FPGA dev card too, which is way more powerful... but, it costs 500 bucks - so I used the DE0-Nano instead (as I had one already) - still, the benefit of open-source, is you can port it to whatever you like :)

In theory I think you could raise the design to 16-bit at around 80 MSPS or even more if you expanded the FPGA->FX3 bus to 32-bits, but I wanted to keep the cost and complexity down for the first version. There's very little information on the web about building your own high-speed DAQ; so perhaps this project will help inspire some other people (since it could be used as an oscilloscope or SDR with some front-end modification).

Here's a picture of the 1_2 version of the board just to give you an idea of what it looks like (the Kicad files on Github are for 1_3 which I am going to build this weekend):

Attachment:
File comment: Domesday Duplicator 1_2
DomDup1_2.jpg
DomDup1_2.jpg [ 119.27 KiB | Viewed 1693 times ]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 05 Nov 2017, 14:59 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
I've now built the new version of the board and the results look very promising. I was able to convert 28.8MSPS captures to 8-bit and then successfully run them through ld-decode :)

I've prepared some sample captures from a PAL laserdisc and PM'd a link to you. The files are 32MSPS 16-bit signed samples (scaled from the native ADC resolution of 10-bits).

I've also started to document the project; you can find the project page from the following link:

http://www.domesday86.com/?page_id=978

The project page also contains some oscilloscope traces of the RF front-end and a number of FFT analysis results from the sampling.

/Simon
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 05 Nov 2017, 17:25 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Sweet!

And I got the PM and downloaded the 32mhz samples. I'm cleaning up the PAL version now and should have decodes soon(tm).
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 05 Nov 2017, 17:46 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Out of curiosity I took one of the samples and used Adobe Audition to down-sample it to 28.8 MHz, then to 8-bits (just to check if the sample was valid), and ld-decode gave me a good looking frame. So I'm pretty confident the samples are good :)

Attachment:
frame-9.jpg
frame-9.jpg [ 146.55 KiB | Viewed 1670 times ]


The only thing I'm thinking about doing to the ADC hardware is adding a 16MHz low pass filter; but I'm not convinced it's required. Do you know if the original TV card had a filter? I guess doing some FFT analysis of the RF tap using my oscilloscope will help make the decision though.

I'm looking forward to actually getting some complete captures, I've been working on this DAQ for months - the high data-rates make it quite challenging to design :)

BTW: Once the board design is complete I will redesign the PCB for professional production and get a few PCBs made as I need at least 3 for the Domesday86 project. I will also make a complete board and send it to you as a thanks for all the help you've given.
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 06 Nov 2017, 15:59 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Just pushed a first and not-correct-yet 32mhz decoder and TBC code. Got the TBC working right for black and white but the color phase is all wrong.

Can you put up ~4-8MB of the resampled RF (for the test pattern as well if possible) for comparison please?

I think the RF test point usually has some LPF circuitry in it (the relevant circuit to me is on the RF test point on page 18 on http://manuals.lddb.com/LD_Players/Pion ... l_Scan.pdf , near Q152's Inverter), but it looks kinda odd to me - but it's duplicated on both the testpoint and non-testpoint side. Since you said there was aliasing etc I suspect the capture card doesn't have a filter (but some other implementations like the ATI card might...)

And thanks very much - I was going to PM you about buying a board sooner or later!


Last edited by happycube on 06 Nov 2017, 16:59, edited 1 time in total. _________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 06 Nov 2017, 16:27 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Do you want them resampled to 16 bit signed 28.6364MSPS or 8 bit unsigned 28.6364MSPS?

As a side-note I think you can do the same trick with Audacity (which is GPL open-source); if you import the sample as a 16-bit signed raw (1 channel) sample at 32000KHz and then resample to 286363Khz then the maths works just as well (the audio software just thinks it's a really long sample :) ). That's all I did with Adobe Audition to check the contents were valid.

Also, would it help if I made the samples unsigned like the original 8 bit samples? I decided on signed 16-bit because it makes it easier to post-process the samples in audio software (both Audacity and Audition expect 16-bit samples to be signed).

To get the file from 16 bit unsigned to 8 bit signed I just wrote a really simple C program. You give it ldcapture16.raw and it outputs ldcapture8.raw in response:

// LDconvert - main.c

#include <stdio.h>
#include <stdint.h>

#define BUFFER   4096

int main()
{
   char buffer[BUFFER];
   char bufferout[BUFFER / 2];
   
   uint16_t bytesRead = 0;
   uint16_t counter;
   int16_t readValue;
   uint64_t outpointer = 0;
   
   printf("Converting 16-bit signed sample to 8-bit unsigned...\r\n");
   
   // Read the 16-bit values from the raw capture file
   FILE *ptr_myfile;
   FILE *ptr_myfileout;

   ptr_myfile=fopen("ldcapture16.raw","rb");
   ptr_myfileout=fopen("ldcapture8.raw","w");
   if (!ptr_myfile)
   {
      printf("ERROR: Unable to open file!\r\n");
      return 1;
   }

   // Read a buffer full of data
   while(bytesRead = fread(&buffer,1, sizeof(buffer), ptr_myfile)) {
      for (counter = 0; counter < bytesRead; counter += 2) {
         
      readValue = ((unsigned char)buffer[counter+1] << 8) | (unsigned char)buffer[counter];

      
      // Convert 16-bit signed to 8-bit unsigned
      readValue = readValue / 256;
      readValue = readValue + 128;
      bufferout[outpointer] = (uint8_t)readValue;
      outpointer++;
      }
      
      fwrite(&bufferout, 1, sizeof(bufferout), ptr_myfileout);
      outpointer = 0;
   }
   

   fclose(ptr_myfile);
   fclose(ptr_myfileout);
   
   printf("Done\r\n");
   
   return 0;
}
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 06 Nov 2017, 17:19 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
I've PM'd you exactly the same test samples as before both in 28MSPS 16-bit signed and 28MSPS 8-bit unsigned. Hopefully they will work for you; if not, let me know and I'll re-run the conversions.

/Simon
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 06 Nov 2017, 17:29 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Thanks! Downloading it now... I'm out of time for this morning (#dayjob awaits) but I'll look at it more tonight/tomorrow morning.

Made a bit more headway on 32mhz/16bit processing, but the color phase is all wrong right now so being able to compare it to the 28/8 code is quite helpful.
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 06 Nov 2017, 20:18 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Quote:
I think the RF test point usually has some LPF circuitry in it


The schematic is a bit hard to read, but it looks like a 47R resistor and a 100nF capacitor, which would act as a low-pass, but not a very effective one with so few components. There is a third component, but I can't tell what it is; at a guess I would think perhaps a zener diode to limit the voltage. It's also not clear if the RF test-point is connected to the filter; there's a 47R resistor (50Ohms impedance) but I don't see a 'node' - it's also unlikely that the test point is connected to the collectors of the two amplifier transistors, the RF path flows into the base of Q152. I think it more likely that the LPF is not a LPF at all, but power conditioning... R193 limits the current, the zener limits the voltage and the 100nF capacitor is for decoupling. Noise into the amplifier would be an issue at such an early stage of the analogue processing; so it's good engineering to be careful with the power.

The cut off frequency with those components (if it really is an LPF) would be around 33MHz - which also leads me to believe it's not a signal filter.

During testing I got far worse results with older discs than newer - and the picture degradation was worse on the sampled frames than the actual video output from the player. This leads me to believe that scratches and other imperfections cause high-frequency noise in the signal and therefore much more aliasing noise in the sample which (unlike an analogue system) can't be filtered out after the sample is made. I've ordered some components to make a butterworth LC-filter and tomorrow (once I have them) I'll construct a test circuit with them and perform some bode analysis to see what the real-world effect would be. I'm almost convinced now that an aliasing filter is a good ides - although any filter will cause attenuation of the signal, and that means I'll need to compensate for it across the opamp.

I'm really (really!) happy that you're making progress with the software - I can't wait to see the effect of the higher-resolution sampling :)
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 07 Nov 2017, 09:05 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
That makes sense - and the filtering in ld-decode-pal[32] isn't right yet especially for this type of capture.

I've made some headway tonight - got a coherent color signal after some hacking, but there's herringbone noise in the color decoder. I think chroma noise can be reduced a bit once it does work, though...

edit Nov 7 AM: Still don't have it playing nice with the comb filter, but...
Image
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 07 Nov 2017, 22:21 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Test image looks good :)

I've been doing some more work on the hardware around the anti-aliasing and have now implemented a low-pass filter. Running some tests (sampling at 32MSPS and then down-sampling to 28.6MSPS 8-bit) the new filter seems to significantly improve the colour noise in the image. I'm starting to think that some of the other issues you're seeing are being caused by the aliasing errors.

I captured a scene from a disc and put it up on my site (since the forum limits you to 150K JPEGs for some reason). Here's a link (this is the unaltered output from ld-decode):

http://www.domesday86.com/wp-content/uploads/2017/01/after.png

If you compare it to the similar image I posted above, the difference is quite obvious I think.

Based on these results I think it would be a good idea to recreate the test samples I made for you with the filter in place. I don't have time tonight, but I'll try to get it done tomorrow afternoon and the PM you a link to the new versions.
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 05:16 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Looks nice! And for test samples - I only need ~10MB of each still frame, and if you have anything with large blocks of solid color that'd be wonderful. (I forget what the Jason disk has, since I don't have a copy.)

Also, do you think the 40mhz ADC can be pushed to ~42.95mhz? 12fsc is a nice 'round' number for NTSC captures.
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 05:56 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
I tried cloning the latest github and running it against the test files but I seem to have some issues even getting the new code running (I'm not really very familiar with python).

The #!python change to the top of the file causes "python: bad interpreter: No such file or directory" (ubuntu 16.04) - the previous version had a different path which worked. If I change this back I then get:

SyntaxError: Non-ASCII character '\xcf' in file ./ld-decoder-pal32.py on line 17, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Looks like you have unicode characters in the file, but no encoding directive?

What environment are you using to develop the code? Perhaps it's easier if I just set up the same environment :)

No rush on this; I just thought I'd mention the issue.
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 06:05 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Quote:
Looks nice! And for test samples - I only need ~10MB of each still frame, and if you have anything with large blocks of solid color that'd be wonderful. (I forget what the Jason disk has, since I don't have a copy.)

Also, do you think the 40mhz ADC can be pushed to ~42.95mhz? 12fsc is a nice 'round' number for NTSC captures.


I'll grab some more of the test frames from the Jason disc and try to cut them down to smaller files there are some more simple test cards I think. I'll post them later today.

The ADC maximum sampling rate is 40MSPS, but the more you push it, the more critical noise becomes. There's no way to overclock it to anything above 40MSPS... having said that; the design can be modified with an even faster ADC (and even 16-bit resolution) by changing the ADC, but that can lead to other issues (especially noise from the digital side of the ADC).

I would guess that resolution is more important than speed though (since the design is already way over the required nyquist) - if the current design proves sub-optimal once you test with NTSC then I can look into modifying it. I have a friend with over 2000 laserdiscs who wants backups (smally_uk on this forum), so there are drivers outside of my Domesday project :)
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 06:15 
Absolute fan
Absolute fan
User avatar

Joined: 18 Apr 2012, 18:02
Posts: 1555
Location: United States
Has thanked: 46 times
Been thanked: 44 times
Thanks!

I'll change the #!python to #!python3 - I'm using a fresh install of Anaconda. And I'm sure 32mhz@10bit will be fine, I'm just thinking a bit greedily :)

And that's a huge collection!
_________________
Happycube Labs: Where the past is being re-made, today. [meep!]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 07:13 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
You have a PM with the test cards (I now need to drive at a billion MPH to get to work - but I couldn't wait :) )

Here is test card 1/5 from the Jason disc (I'm limited to one picture per attachment, so I'll add the others as separate posts just so there is a note of what's on the disc)

Attachment:
testcard1.jpg
testcard1.jpg [ 66.07 KiB | Viewed 1554 times ]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 07:13 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Test card number 2:

Attachment:
testcard2.jpg
testcard2.jpg [ 37.02 KiB | Viewed 1554 times ]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 07:13 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Test card number 3:


Attachments:
testcard3.jpg
testcard3.jpg [ 84.98 KiB | Viewed 1554 times ]
Offline
 Profile  
 
 Post subject: Re: (WIP) Laserdisc software image decoder from raw signal
PostPosted: 08 Nov 2017, 07:14 
Honest fan
Honest fan
User avatar

Joined: 11 Sep 2017, 14:24
Posts: 86
Location: Sweden
Has thanked: 1 time
Been thanked: 34 times
Test card number 4:

Attachment:
testcard4.jpg
testcard4.jpg [ 65.48 KiB | Viewed 1554 times ]
Offline
 Profile  
 
Display posts from previous:  Sort by  
 Page 15 of 23 [ 457 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 23  Next


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

cron