Google Earth Engine Python API: Map function over image collection with a list of bands

Question

I have been using the code by Sam Murphy for atmospheric correction of Sentinel-2 images in Google Earth Engine. All goes well and it runs very fast for a single image. What I would like to do is map the following code over an image collection:

output = image.select('QA60')
for band in ['B1','B2','B3','B4','B5','B6','B7','B8','B8A','B9','B10','B11','B12']:
    print(band)
    output = output.addBands(surface_reflectance(band))

I think I need a double map function for this (to avoid using the for-loop here), but have not seen any Python examples in GEE yet.

So far this is what I have come up with:

def atcorrector(image):
    qa = image.select('QA60')
    bands = ee.List(['B1','B2','B3','B4','B5','B6','B7','B8','B8A','B9','B10','B11','B12'])
    def mapper(bands):
        return qa.map(addBands(surface_reflectance(bands)))
    return qa

ImageCollection.map(atcorrector)

However this does not return image with all the bands, so I feel that my nested function does not work properly. I am new to python, so all help is appreciated!

Side-Note: I have already experimented with Sam's second repository for atmospheric correction over image collections, but it runs far too slowly and I would prefer to have a 'server-side' computation using a proposed map function as I have loads of images to process.

PS: Below is the code for the surface_reflectance function, as extracted from Sam Murphy's Repository. It calls on one of his customised classes called "Atmospheric". The model used for atmospheric correction is the Py6S model.

# Package requirement
from Py6S import * 
import datetime
import math
import os
import sys
sys.path.append(os.path.join(os.path.dirname(os.getcwd()),'bin'))
from atmospheric import Atmospheric #Custom-defined class by Sam
import ee
ee.Initialize()

## The Sentinel-2 image collection
studyarea = ee.Geometry.Rectangle(7.8399,59.9273,8.2299,60.1208)#region of interest
S2 = ee.ImageCollection('COPERNICUS/S2').filterBounds(studyarea)\
       .filterDate('2016-06-01', '2016-06-10').sort('system:time_start')

## define metadata
info = S2one.getInfo()['properties']
scene_date = datetime.datetime.utcfromtimestamp(info['system:time_start']/1000)# i.e. Python uses seconds, EE uses milliseconds
solar_z = info['MEAN_SOLAR_ZENITH_ANGLE']
SRTM = ee.Image('USGS/GMTED2010')  # Make sure that your study area is covered by this elevation dataset
alt = SRTM.reduceRegion(reducer=ee.Reducer.mean(), geometry=studyarea.centroid()).get('be75').getInfo() # insert correct name for elevation variable from dataset
km = alt/1000  # i.e. Py6S uses units of kilometers

date = ee.Date(START_DATE)
# the following three variables are called on from the Atmospheric class Sam defined in his GitHub
h2o = Atmospheric.water(studyarea,date).getInfo() 
o3 = Atmospheric.ozone(studyarea,date).getInfo()
aot = Atmospheric.aerosol(studyarea,date).getInfo()

## Create the 6S Object
s = SixS() # Instantiate
# Atmospheric constituents
s.atmos_profile = AtmosProfile.UserWaterAndOzone(h2o,o3)
s.aero_profile = AeroProfile.Continental
s.aot550 = aot

# Earth-Sun-satellite geometry
s.geometry = Geometry.User()
s.geometry.view_z = 0               # always NADIR (I think..)
s.geometry.solar_z = solar_z        # solar zenith angle
s.geometry.month = scene_date.month # month and day used for Earth-Sun distance
s.geometry.day = scene_date.day     # month and day used for Earth-Sun distance
s.altitudes.set_sensor_satellite_level()
s.altitudes.set_target_custom_altitude(km)

# Extract spectral response function for given band name
def spectralResponseFunction(bandname):
    bandSelect = {
        'B1':PredefinedWavelengths.S2A_MSI_01,
        'B2':PredefinedWavelengths.S2A_MSI_02,
        'B3':PredefinedWavelengths.S2A_MSI_03,
        'B4':PredefinedWavelengths.S2A_MSI_04,
        'B5':PredefinedWavelengths.S2A_MSI_05,
        'B6':PredefinedWavelengths.S2A_MSI_06,
        'B7':PredefinedWavelengths.S2A_MSI_07,
        'B8':PredefinedWavelengths.S2A_MSI_08,
        'B8A':PredefinedWavelengths.S2A_MSI_09,
        'B9':PredefinedWavelengths.S2A_MSI_10,
        'B10':PredefinedWavelengths.S2A_MSI_11,
        'B11':PredefinedWavelengths.S2A_MSI_12,
        'B12':PredefinedWavelengths.S2A_MSI_13,
        }
    return Wavelength(bandSelect[bandname])

# Converts top of atmosphere reflectance to at-sensor radiance
def toa_to_rad(bandname):
    ESUN = info['SOLAR_IRRADIANCE_'+bandname]
    solar_angle_correction = math.cos(math.radians(solar_z))# solar exoatmospheric spectral irradiance
    doy = scene_date.timetuple().tm_yday
    d = 1 - 0.01672 * math.cos(0.9856 * (doy-4))# Earth-Sun distance (from day of year)
    multiplier = ESUN*solar_angle_correction/(math.pi*d**2)# conversion factor
    rad = toa.select(bandname).multiply(multiplier)# at-sensor radiance
    return rad

# Calculate surface reflectance from at-sensor radiance given waveband name
def surface_reflectance(bandname):
    s.wavelength = spectralResponseFunction(bandname)  # run 6S for this waveband
    s.run()
    # extract 6S outputs
    Edir = s.outputs.direct_solar_irradiance             #direct solar irradiance
    Edif = s.outputs.diffuse_solar_irradiance            #diffuse solar irradiance
    Lp   = s.outputs.atmospheric_intrinsic_radiance      #path radiance
    absorb  = s.outputs.trans['global_gas'].upward       #absorption transmissivity
    scatter = s.outputs.trans['total_scattering'].upward #scattering transmissivity
    tau2 = absorb*scatter                                #total transmissivity
    # radiance to surface reflectance
    rad = toa_to_rad(bandname)
    ref = rad.subtract(Lp).multiply(math.pi).divide(tau2*(Edir+Edif))
    return ref

Answer 1

To apply the surface_reflectance function as you want, just modify the code as follow:

def atcorrector(image):
    qa = image.select('QA60')
    for band in ['B1','B2','B3','B4','B5','B6','B7','B8','B8A','B9','B10','B11','B12']:
        print(band)
        qa = qa.addBands(surface_reflectance(band))
    return qa

ImageCollection.map(atcorrector)

As you see, this code just copies the code that you've made for a single image. The for loop works without any issues in Python API (in JavaScript API, I would not recommend using this). In case you don't want to use the for loop, just change the code a bit:

def atcorrector(image):
    qa = image.select('QA60')
    bands = ee.List(['B1','B2','B3','B4','B5','B6','B7','B8','B8A','B9','B10','B11','B12'])

    def mapper(band):
        qa = qa.addBands(surface_reflectance(band))
        return band

    bands.map(mapper)
    return qa

ImageCollection.map(atcorrector)

The map function of ee.List (or any ee object that has the map function) could be a replacement for the for loop.

Hope this helps.