Canon Patent Application: Stacked sensor with memory patent

In this patent, Canon describes an image sensor that is stacked and contains memory cells.

Canon is looking for ways to improve automatic HDR (stacking images for the purpose of a wider dynamic range), without dramatically increasing the amount of memory required.  To do image alignment, it's necessary to store each image to process it separately.  Canon is developing a method that reduces the amount of memory (ie: circuit scale) required as resolution increases.

Patent Document 1 discloses correcting image blur in an added image by aligning the images when adding the images.
However, in the configuration of Patent Document 1, Since it is necessary to provide the image sensor with a frame memory for adding images, the circuit scale increases. Since the image resolution (the number of pixels per frame) is increasing, the capacity of the frame memory is expected to increase in the future. An   object of the present invention is to provide an image pickup device capable of easily obtaining a corrected image while suppressing an increase in circuit scale.

Now comes the fun part of this - this image sensor is also a photon counting image sensor (PCIS) which uses avalanche photodiodes (APD) to count each photon that hits the sensor.  If you been following us, you'd see that we have been exclusively discussing Canon's R&D and patent applications in photon counting and/or APD based sensors.

The above-described object is an imaging device in which a plurality of pixels are arranged, and each of the plurality of pixels includes a light-receiving element that can detect the incidence of a single photon, and a counter that counts pulses included in the signal. The image pickup device further includes a control unit that switches a signal of the light receiving element supplied to the counter based on the detected blur or replaces the count value of the counter with the count value of the counter included in another pixel.

Could this be the next big thing? Why are we talking about this anyways? Photon Counting sensors have their issues (usually surrounding incoming photons coming too quickly ie: alot of light), so it's not a perfect science.  If you look back through our patents, you can see that Canon even talks and is working on solutions for that.  A photon counting sensor is the ultimate low light sensor.  It's actually used for those reasons, atypically industrial and scientific reasons where there is a need to have accurate low level light data recorded.

Now, this doesn't mean that Canon is going to put a PCIS/APD sensor in the 1DX Mark III, or the next profession EOS R camera, but it is an area of active Canon research.  It may or may not end up in actual products, or may be simply an internal use product - but it is interesting.

Japan Patent Application 2019-165324