9 minute read

Camera Sensors: What Are They and How Do They Work?

Learn about digital camera sensors and understand the essential role they play in digital photography

Digital cameras are everywhere – from high-end professional equipment used by the media to everyday smartphone cameras, webcams, and even doorbells. At the heart of every single one is a digital camera sensor, also known as an image sensor. Without this vital piece of technology, digital cameras as we know them today simply would not exist.

But what are camera sensors and how do they work? We aim to outline the basics behind the most common type of camera sensor and explain how this ever-crucial technology has evolved.

What is a Camera Sensor?

At the most basic level, a camera sensor is a solid-state device that absorbs particles of light (photons) through millions of light-sensitive pixels and converts them into electrical signals. These electrical signals are then interpreted by a computer chip, which uses them to produce a digital image.

While there are a number of different types of camera sensor, by far the most prevalent is the complementary metal-oxide semiconductor (CMOS) sensor, which can be found inside the vast majority of modern digital cameras.

This includes smartphones, compact cameras and mirrorless interchangeable-lens cameras (ILCs).

Cross section of a camera sensor pixel. For illustrative purposes only.

Figure 1: Cross section of a camera sensor pixel. For illustrative purposes only.

How Does a CMOS Sensor Work?

A CMOS sensor is made up of a grid of millions of tiny pixels. Each pixel is an individual photosite, often called a well (see Figure 1).

When photons enter the photosite, they hit a light-sensitive semi-conductor diode, or photodiode, and are converted into an electrical current that directly corresponds to the intensity of the light detected.

This signal is amplified on-pixel, then sent to an analog-to-digital converter (ADC), which converts it into digital format and sends it to an image processor.

The image processor is able to read these digital signals collectively and translate them into an image, because each pixel is assigned an individual value, depending on the intensity of light it was exposed to.

As you can see in Figure 1, because the conversion and amplification processes happen on-pixel, the transistors, wiring, and circuitry have to be included in the spaces between each photosite.

To minimize the amount of light bouncing off this circuitry, a microlens is placed on the top of each pixel to direct the light into the photodiode and maximize the number of photons gathered.

You may have also noticed the inclusion of a color filter in Figure 1. The reason for this is that pixels detect light, not color, so a camera sensor by itself can only produce black & white images.

In order to create color images, a color filter array needs to be added.

What Is a Color Filter Array?

A color filter array is a pattern of individual red, green, and blue color filters arranged in a grid – one for every pixel. These filters sit on top of the photosites and ensure that each individual pixel is exposed to only red, green, or blue light.

This means the image processor can assign color to each pixel readout and in turn produce color images.

Color filter arrays come in different patterns, the most common of which is the Bayer filter array.

The Bayer Filter Array

The Bayer color filter array

Figure 2: The Bayer color filter array. For illustrative purposes only.

The Bayer filter array (see Figure 2) is made up of a repeating 2×2 pattern in which each set of four pixels consists of two green, one red, and one blue pixel. This equates to an overall split of 50% green, 25% red, and 25% blue.

The reason there is a higher frequency of green filters is because the filter array has been designed to mimic the human eye’s higher sensitivity to green light.

While the Bayer filter array is effective, it does create a problem – the appearance of an unwanted effect called moiré.

What Is Moiré?

Moiré occurs when photographing a uniform pattern equal to or of a higher resolution than the camera’s sensor.

Common instances in which moiré can be seen are when photographing brick walls from a distance, fabrics, or display screens. If the pattern being photographed misaligns with the grid created by the color filter array, strange effects appear, as illustrated in Figure 3.

This was a major problem in the early days of digital photography when sensor resolutions were lower. However, with sensors now enjoying much higher resolutions, moiré is less common.

One way to prevent moiré is by adding an optical low-pass filter to the sensor. Another is to use a different color filter array.

What Is an Optical Low-Pass Filter?

An optical low-pass filter – also known as an anti-aliasing filter – is a filter placed in front of a camera sensor to slightly blur the fine details of the scene being exposed, thereby reducing its resolution to a level below that of the sensor.

Optical low-pass filters can certainly be effective in preventing moiré, but this comes at a cost.

Although the effects of the filter are so slight that they are invisible to many everyday photographers, blurring inevitably equates to a reduction in sharpness. This is undesirable for many professionals, and is one of the reasons Fujifilm developed the X-Trans color filter array.

X-Trans Color Filter Array

The FUJIFILM X-Trans color filter array

Figure 4: The FUJIFILM X-Trans color filter array. For illustrative purposes only.

The X-Trans color filter array was introduced in 2012 with the release of FUJIFILM X-Pro1.

Made up of approximately 55% green, 22.5% red, and 22.5% blue filters, it creates similar proportions of red, green, and blue pixels as the Bayer array. But it uses a more complicated 6×6 arrangement, comprised of differing 3×3 patterns.

Using a less uniform pattern helps reduce moiré, eliminating the requirement for an optical low-pass filter and in turn creating sharper images.

Sensor resolutions have risen dramatically since the 16-megapixel X-Trans CMOS sensor in X-Pro1, making it less likely for moiré to occur. As a result, optical low-pass filters have all but disappeared – though increased image sharpness is not the only potential advantage of the X-Trans color filter array.

Every vertical and horizontal line in an X-Trans CMOS sensor includes a combination of red, green, and blue pixels, while every diagonal line includes at least one green pixel. This helps the sensor reproduce the most accurate color.

Additionally, the less uniform pattern is closer to the random arrangement of silver particles on analog photographic film, which contributes to Fujifilm’s much-loved film-like look.

Demosaicing

As covered above, a single pixel can only record a single value. But if you zoom into a digital image, each individual pixel can contain a mixture of colors, rather than just the red, green, or blue allowed by the color filter array.

So, how are the other colors added? And how does the camera know the correct amount to use?

The answer is a process called demosaicing, in which a demosaicing algorithm predicts the missing color values for an individual pixel based on the strength of the color recorded by the pixels that surround it.

This is done automatically by the camera’s built-in processor, which then turns it into a viewable image file format such as JPEG or HEIF.

In many cases, such as photographing on a smartphone, that is the end of the process. However, most mirrorless cameras have the ability to save images in RAW format, providing photographers with more options.

As the name suggests, a RAW file contains the raw image data before any demosaicing has taken place. This allows photographers to demosaic images using external software such as Capture One.

Different types of software use distinct demosaicing algorithms, each offering unique aesthetics. An obvious advantage of this is that photographers can choose their personal preference, but the benefits of creating in RAW format extend much further.

Advantages of RAW Files

File types such as JPEG and HEIF are designed to make image files easily portable, so significant compression takes place to achieve the smallest possible file sizes.

During the compression process, a large amount of tonal and color information read by the sensor is lost. Less information means lower quality and, in turn, restricted freedom to edit.

As a result, RAW files contain a wider dynamic range and broader color spectrum, which allows for more effective exposure correction and color adjustments.

Evolution of the CMOS Sensor

While the basic operation of the CMOS sensor has remained fundamentally the same throughout its history, its design has evolved to maximize efficiency and speed.

Back-Side Illuminated Sensor

Cross section of a front-side illuminated vs back-side illuminated CMOS sensor. For illustrative purposes only.

Figure 5: Cross section of a front-side illuminated vs back-side illuminated CMOS sensor. For illustrative purposes only.

In the case of the original front-side illuminated (FSI) sensor design, all the wiring and circuitry necessary for storing, amplifying, and transferring pixel values runs along the borders between each pixel. This means light has to travel through the gaps to reach the photodiode beneath.

As its name suggests, the back-side illuminated (BSI) sensor flips this original design around so the light is now gathered from what was its back side, where there is no circuitry.

By removing the obstruction caused by the circuitry, a greater surface area can be exposed to light, allowing the sensor to gather more photons and subsequently maximize its efficiency.

The result is increased sensitivity, lower noise, and ultimately higher-quality images.

Stacked Sensor

Cross section of a stacked CMOS sensor. For illustrative purposes only.

Figure 6: Cross section of a stacked CMOS sensor. For illustrative purposes only.

While the BSI sensor design increases quality, the stacked sensor is all about increasing speed.

Until the introduction of the stacked sensor, CMOS sensors operated on a single layer. This meant the signal readouts from each pixel had to travel along strips of wiring all the way to the outside of the sensor before they were processed.

With stacked sensors, these processing chips have been added to the back of the sensor, essentially creating a ‘stack’ of chips sandwiched together.

By stacking them in this way, the distance the pixel values have to travel is drastically reduced, resulting in much faster processing speeds.

For example, the X-Trans CMOS 5 HS stacked sensor found in FUJIFILM X-H2S enjoys four times the reading speed of its predecessor and 33 times the reading speed of the original X-Trans CMOS sensor featured in X-Pro1.

What’s more, without the problem of obstructing light entering the sensor, it’s possible to keep stacking additional chips, offering huge potential for future developments.

Conclusion

Like any technology, camera sensors have come a long way in the past decade alone, and look to continue this development into the future.

With the move to back-side illumination enabling much higher resolutions and stacked sensors increasing readout speeds so significantly, recent developments amount to nothing short of a revolution in CMOS camera sensor technology.

The door is now open for huge future advances, equipping CMOS sensors with capabilities that simply weren’t possible only a few years ago.

Learn more by exploring the rest of our Fundamentals of Photography series, or browse all the content on Exposure Center for education, inspiration, and insight from the world of photography.


CREATE/SEND/SHARE

see your image on our Social Channels

Share your image and story with us for a chance to be featured.

Success.
Please select an image
What would you like to share about this image and why would you like to share it?

Consumer Submission Terms and Conditions:

MY FUJIFILM LEGACY MEMORY
1. AGREEMENT TO THESE TERMS AND THE FUJIFILM PRIVACY POLICY: BY UPLOADING A SUBMISSION (as hereafter defined) AND CLICKING YOUR AGREEMENT TO THESE TERMS ON THE SITE, YOU ARE AGREEING TO BE BOUND, WITHOUT LIMITATION OR QUALIFICATION, TO THESE TERMS AND CONDITIONS AND THE PRIVACY POLICY, AVAILABLE AT https://www.fujifilm.com/privacy_policy/?_ga=2.188005036.507021620.1563365952-1582032552.1552994675. Any information contained in a Submission, including image, contact information such as physical address and email address, and any other personal information submitted, will be used in accordance with FUJIFILM North America Corporation’s (“FNAC”) Privacy Policy and these Terms & Conditions. Each Submitter acknowledges that submission pursuant to these Terms and Conditions constitutes the necessary and proper permission for FNAC’s use of his or her Submission pursuant to these terms and the Privacy Policy. In addition, by uploading a Submission on the Site, each Submitter represents and warrants that he or she is at least 13 years of age. Yes, I’d like to receive more information regarding FUJIFILM products and services via e-mail (e.g. by newsletter or other marketing communication). I also agree that FUJIFILM North America can use my contact data in accordance with its Privacy Policy to inform me about FUJIFILM products and services by e-mail.

2. IMAGE AND PHOTO DESCRIPTION SUBMISSION REQUIREMENTS:
A. Each photo submission (each, along with the required submission form and any provided description, a "Submission") that you as the Submitter determines to submit to FNAC must be uploaded to the submission website at http://www.fujifilm-x.com/en-us/ShareMyFujifilmLegacy (the “Site”) and accompanied by a fully completed submission form located on the Site. Each submitted photo may be no larger than 10 MB, no longer than 2000 pixels in the longest dimension, and it must be formatted as a JPEG, TIFF, GIF, PNG, or BMP. Any description you provide relating to the image may be no more than 1500 characters, including spaces. The Submission may reflect any topic or subject matter, but must be the original work of the providing Submitter, and must depict an original photographic creation. The Submission must have been captured using a FUJIFILM X Series camera and lens (i.e. any FUJIFILM XF or XC lens) or GFX System camera and lens (i.e. any GFX lens).

B. No commitment is made by FNAC to review, consider or post any Submission, and FNAC reserves the right, in its sole discretion, to deem any Submission as inappropriate, unworthy, or unsuitable for posting for any reason, and at any time with no obligation to inform, explain, or justify its reasoning


C. The Submission(s) must not infringe upon the copyrights, trademarks, rights of privacy, publicity or other intellectual property or other rights of any third party person or entity, and by uploading a Submission you represent and warrant that: (a) the Submission is your personal, sole, and original creation and has not been derived from, reproduced, or copied in whole or in part from any work by a third party person or entity; (b) the Submission does not violate or infringe any intellectual property right of any kind, including but not limited to any copyright, trademark, trade name, publicity right, or other proprietary right of any person or entity; (c) the Submission is your personal, sole, and exclusive property; (d) any person(s) depicted in the image has given his or her approval for the submission; and (e) no person or property depicted in the image is depicted in a misleading, misrepresentative, or false fashion;


D. The Submission(s) and its title and description are subject to the following content guidelines, and may not, in any instance, in FNAC’s judgment:


(i) Contain, depict, reflect, promote or endorse nudity, profanity, obscenity or pornography; content that is sexually explicit or suggestive, violent, derogatory, disparaging, or defaming; or potentially hazardous content, behavior or activities;

(ii) Contain, depict, reflect, promote or endorse the sale, distribution, possession, use, or abuse of alcohol, drugs, tobacco, firearms, knives or other weapons; or any particular social or political agenda, issue or message;

(iii) Appear to duplicate or replicate a previously received Submission;

(iv) Contain, depict, reflect, promote or endorse any third party owned trademarks, logos, copyrights, images, or trade dress (such as clothing bearing a distinctive logo, a third party’s artwork, or materials from movies or published in other media) including third party products;

(v) Contain, depict, or reflect any identifying information relating to any person depicted, (such as license plate numbers, personal names, email addresses or street addresses), and including any likeness or other information of famous individuals (such as past or present celebrities or public figures, or individuals impersonating them);

(vi) Contain, depict, reflect, promote, endorse or communicate content inconsistent with the reputation and corporate image FNAC and/or FUJIFILM Corporation wish to communicate; and/or

(vii) Violate any law, or reflect, advocate or promote any activity that may violate any law.

Submissions determined by FNAC to violate any of the above content guidelines are subject to rejection. FNAC reserves the right to take all necessary measures to remove any given Submission as it deems appropriate in any given instance, and/or to change any Submission title or description provided at any time and for any reason.


E. If the Submission(s) contains, reflects or incorporates any materials, content or elements that are subject to the rights of third parties, the Submitter is responsible for obtaining, prior to providing the Submission, any and all releases, permissions and consents necessary to permit the use and posting of the Submission by FNAC and its parent and affiliate companies (including without limitation, FUJIFILM Corporation). This includes obtaining the written release, permission and consent to use any name, image, likeness or property of any person who, or items that appear in or are identifiable in the Submission(s). Where a minor appears in or is identifiable in any given Submission, the Submitter is responsible to put a written release, consent and permission into place with the appropriate parent or guardian of such minor. FNAC reserves the right to request documentation reflecting these releases, permissions and consents from any Submitter at any time, and to undertake measures to remove the posting of any given Submission(s) pending receipt of such documentation.


3. LICENSE FOR FNAC TO POST AND POSTING GUIDELINES: Upon submitting a Submission, each Submitter grants to FNAC, its parent and affiliate companies, a royalty free, irrevocable, perpetual and worldwide right and license (but without any right of sublicense) to modify, adapt, and reproduce the Submission, and to use and display the Submission, in whole or in part, either (i) on the FNAC Instagram (@fujifilmx_us), Facebook (http://www.facebook.com/FujifilmXUS), or Twitter (@FujifilmX_US) Account, as a social media post; or (ii) on a FNAC or parent/affiliate company website (such as, without limitation, www.fujifilm-x.com), including without limitation, as a homepage banner. Submitter understands that any use made of the Submission will be commercial in nature, and may be in connection with the promotion of a FUJIFILM branded product. This right and license is granted in return for FNAC’s review and consideration of the Submission and in return for the potential opportunity for publicity resulting from FNAC’s posting and tagging of the Submission, and without any right or expectation of the Submitter to receive any further compensation (directly or indirectly) of any kind. FNAC and its parent and affiliate companies shall have no obligation to provide further notice to, or to obtain any further approval from, the Submitter. All Submission postings to be made by FNAC will be subject to the following posting guidelines:


A. If FNAC posts any given Submission, FNAC will use commercially reasonable efforts to credit the Submission on Facebook, Instagram and Twitter by tagging the Submitter’s handle as provided to FNAC through the Site. FNAC will credit each Submission on the FNAC website with the Submitter’s first name and last initial, and Instagram handle if and as applicable. If city and state information is provided, FNAC may reference that information as well in connection with any given Submission posting. Note that FNAC does not guarantee that any given Submission will be posted.


B. FNAC is not responsible for any unauthorized third party use of any Submission. Without limitation, this includes any copying, reproduction, distribution, re-postings, forwarding, linking to, or re-publication made by any third party at any time.


C. FNAC shall have the right to modify each Submission, including but not limited to, altering, and editing any provided description. By uploading a Submission, each Submitter agrees that he or she shall have no right of approval, no claim to additional compensation and no claim (including without limitation, claims based upon invasion of privacy, defamation, right of publicity or copyright) arising out of any use, alteration, modification, editing of any or all of the Submission.


4. FUJIFILM SITE CONTENT AND PROPERTY: The trademarks, service marks, trade names, logos, designs, images, and content at any time displayed on the Site, including but not limited to FUJIFILM, INSTAX, FinePix, BUILD YOUR LEGACY, any other trademarks, logos, designs, and intellectual property and the FNAC posted content and images, shall at all times be the sole property of FNAC and/or its parent and affiliates, or shall be the property of third parties (such as any given Submitter) used by FNAC with any necessary and proper permissions in place. Nothing contained on the Site should be construed as granting, by implication, estoppel, or otherwise, any license or right for any Submitter, or any third party, to use any trademark, service mark, logo, design, trade name, product name, content or image displayed on the Site without the prior written permission of FNAC. For absolute clarity and without limitation, no Submitter or third party is permitted to post, display, publish, forward, link to or re-publish any images of FNAC or any Submission(s) of any other party unless specifically authorized to do so in writing by FNAC. Neither FNAC nor its parent or affiliate companies are sponsored, endorsed or administered by, or otherwise affiliated or associated with, Instagram, Facebook, Twitter or any other social media platform.


5. NO REPRESENTATIONS AND WARRANTIES, AND LIMIT OF LIABILITY:

A. FNAC DISCLAIMS ALL WARRANTIES AND REPRESENTATIONS RELATING TO THE SITE AND TO ANY IMAGES OR CONTENT POSTED ON THE SITE, EXPRESS AND IMPLIED, TO THE GREATEST EXTENT PERMITTED BY LAW. FNAC ALSO ASSUMES NO RESPONSIBILITY, AND WILL NOT BE LIABLE FOR, ANY DAMAGE TO, OR VIRUSES THAT MAY INFECT, ANY SUBMITTER’S OR SITE VISITOR’S COMPUTER EQUIPMENT OR OTHER PROPERTY, OR FOR ANY LOSS OR CORRUPTION OF ANY MATERIALS, IMAGE, CONTENT, OR DATA ON ACCOUNT OF ACCESS TO, USE OF, OR BROWSING IN THE SITE, OR RELATING TO OR BY REASON OF A SUBMITTER’S SUBMISSION OF ANY MATERIALS, IMAGE, CONTENT OR DATA TO THE SITE.

B. NOTWITHSTANDING THE FOREGOING OR OTHERWISE ANY OTHER PROVISION HEREOF, IN NO EVENT SHALL FNAC'S TOTAL LIABILITY TO ANY SUBMITTER OR SITE VISITOR FOR ALL DAMAGES, LOSSES, CLAIMS, LIABILITIES AND CAUSES OF ACTION (WHETHER IN CONTRACT, TORT OR OTHERWISE) EXCEED THE AMOUNTS PAID BY SUCH SUBMITTER OR SITE VISITOR TO FNAC, IF ANY, IN CONNECTION WITH SUBMITTING MATERIALS, IMAGES, CONTENT OR DATA TO THE SITE, OR IN ACCESSING THE SITE, AS APPLICABLE. IN NO EVENT WILL FNAC BE RESPONSIBLE IN ANY INSTANCE FOR ANY INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS, LOST OPPORTUNITIES, OVERHEAD OR ADMINISTRATIVE EXPENSES, LOSS OF USE, OR DAMAGE TO REPUTATION OR GOODWILL) EVEN IF APPRISED OF THE LIKELIHOOD OF SUCH DAMAGES.


6. COPYRIGHT INFRINGEMENT AND TAKE-DOWN PROCEDURE: If you believe in good faith that any material, image, or content appearing on the Site infringes your copyright, then you (or your agent) may send us a written notice that complies with this provision, requesting that the applicable image, or content be removed. Note that by submitting this written notice, you acknowledge and agree that FNAC may forward your written notice (to include your, or your agent’s, personal contact information) and any related communications received by FNAC to the original Submitter of the allegedly infringing content. Furthermore, by submitting this notice, you acknowledge and agree that FNAC’s sole responsibility to you upon receipt of such notice will be to remove the allegedly infringing image or content, and you acknowledge and agree that you will have no further right or remedy against, or as to, FNAC for its original posting of the Submission on the Site, although you may have other rights and remedies as to the Submitter.
Your infringement notice must be sent to FNACE[email protected], and must include the following information:


A. A copy of your specific image or copyrighted work that you are claiming has been infringed;
B. identification of the FNAC posted image or content that is claimed to be infringing or the subject of infringing activity, and information reasonably sufficient to allow FNAC to locate such image or content on the Site;
C. the name, address, telephone number, and e-mail address of the complaining copyright owner;
D. a statement that the complaining party has a good faith belief that use of the image or content by the original Submitter is not authorized by the copyright owner, its agent, or the law; and
E. a statement that the information in the notification is accurate, and under penalty of perjury, that the complaining party is authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.
F. a physical or electronic signature of a person authorized to act on behalf of the copyright owner.
Within ten (10) business days of receipt of a written notice meeting the above-referenced requirements, FNAC will take down the allegedly infringing image or content, and notify the Submitter of such image or content of your claim / infringement allegation. FNAC will not re-post the removed, allegedly infringing image or content until it receives written notice from the complaining party that the dispute has been resolved. The process outlined in this provision is the exclusive process to be used to complain to, or otherwise advise, FNAC about any infringement of your rights by any Submitter or any Submitted image and/or content posted on the Site.
7. PRIVACY POLICY: Personal information contained in a Submission or otherwise communicated to FNAC will be used in accordance with FNAC’s Privacy Policy, which can be found here: [INSERT LINK TO PRIVACY POLICY]. In addition, and as stated above in Section 6, your contact information may be shared to a party complaining of infringement. By uploading a Submission, you hereby agree to the Privacy Policy.

8. MISCELLANOUS TERMS:

A. By uploading a Submission, a Submitter is communicating with FNAC electronically. Each Submitter consents to receive communications relating to the Submission from FNAC electronically, generally by email. FNAC may communicate by e-mail, or by other electronic methods such as through social media communications. Each Submitter agrees that all agreements, notices, disclosures and other communications that FNAC provides electronically satisfy any legal requirement that such communications be in writing.

B. All Submissions are administered by FNAC from its offices in Valhalla, New York, United States of America. These Terms and Conditions are governed and interpreted under the laws of the State of New York without regard to its conflict of laws provisions. Each Submitter agree that any claim or dispute against FNAC arising out of or relating to the use of the Site must be resolved by a court located in the Eastern District of New York, unless otherwise agreed upon by all parties. Any claim or cause of action arising out of or related to a Submission must be filed within nine (9) months after such claim or cause of action arose, regardless of any statute or law to the contrary. In the event any such claim or cause of action is not filed within such 9 month period, each Submitter specifically agrees that such claim or cause of action shall be forever barred.

C. These Terms and Conditions and the Privacy Policy, which are part of these Terms and Conditions and are incorporated herein by reference, set forth the entire understanding and agreement between the Submitter and FNAC with respect to a Submission. Each Submitter acknowledge that any other agreements between with respect to a Submission is superseded and of no force or effect. If any provision of these Terms and Conditions shall be determined to be unlawful, void or for any reason unenforceable by a court or other legal authority of competent jurisdiction, then that provision will be severed from these Terms and Conditions and will be deemed replaced by an equivalent enforceable provision that, as nearly as possible, reflects the intent hereof. The severance of any individual provision of these Terms and Conditions will not affect the validity and enforceability of any remaining provisions.

BY UPLOADING A SUBMISSION, EACH SUBMITTER AGREES
TO BE BOUND BY THESE TERMS AND CONDITIONS
AND THE PRIVACY POLICY.

  • Should be Empty: