Cannot get S2P to work on IARPA challenge data

[ClementPeyrardGSI]

Hi,

I got the S2P pipeline working out of the box on some Airbus/Pleiades data. However, when trying to experiment with the Multi-View Stereo 3D Mapping Challenge data from IARPA (UCSD/WV03/PAN), I cannot get it to work properly (the program runs but the results do not make sense).

The main issue seems to be that the tiles between image A and B of a pair are not cropped from the same geographical extent (the content of each tile pair do not match):

• the tile log often ends with: WARNING: sift.matches_on_rpc_roi: found no matches.
• when matches are found in the pair, the image content do not match:
  

I've tried the following without success:

• cropping each image of the pair to the same extent
• cropping each image and RPC of the pair to the same extent using the crop_rpc utility
• rescaling image B to the same resolution/GSD as image A using the scale_image_and_rpc utility

Is this something you have encountered? Do you have any recommendation or pre-processing that I might have missed in the documentation (e.g. regarding GSD between the images, extent of the rasters)?

Thanks for your great job!

[gfacciol]

Can you upload the config.json you are using including the area of interest?
I think this might be a mismatch between images and rpc.

[ClementPeyrardGSI]

Here it is:
https://www.dropbox.com/s/8wqoj5eyra3xfz9/config_UCSD_WV3_PAN_pair.json?dl=0
Also, I do not specify any utm_bbx, is that an issue ?

[gfacciol]

Hi,
Yes, utm_bbx should be removed.
I have verified that the pipeline works with the json pasted below.
My guess is that the ROI selected in your images

  "roi" : {      "x" : 20000,
                   "y" : 10000,
                   "w" : 3000,
                   "h" : 3000  },

might not be visible in both images. Can you verify this point ? (because s2p doesn't)

Other changes in the json json:

• I didn't find the image "Satellite-Images/UCSD/WV3/PAN/01JAN16WV031300016JAN01185802-P1BS-500647760070_01_P001_________AAE_0AAAAABPABN0_lv1.tif" in our copy of the IARPA dataset. So I replaced it with a different one. Note that we no longer specify the RPC as they are extracted automatically from the GeoTIFF files. Also note that the images are accessed over https.
• I've also added/modified the following tags to improve the performance of stereo matching.

  "matching_algorithm" : "mgm_multi",
  "register_with_shear": true,
  "max_processes": 16, 

Json used in my test

{
  "out_dir" : "iarpa_pair",
  "images" : [
    { "img" : "/vsicurl/http://menthe.ovh.hw.ipol.im/IARPA_data/cloud_optimized_geotif/07JAN16WV031000016JAN07142142-P1BS-500537128030_01_P001_________AAE_0AAAAABPABE0.TIF"},
    { "img" : "/vsicurl/http://menthe.ovh.hw.ipol.im/IARPA_data/cloud_optimized_geotif/05JAN15WV031000015JAN05135727-P1BS-500497282040_01_P001_________AAE_0AAAAABPABR0.TIF"}
    ],
  "roi": {
    "h": 1500.0,
    "w": 1500.0,
    "y": 14000,
    "x": 16000
  },
  "full_img" : false,
  "matching_algorithm" : "mgm_multi",
  "horizontal_margin": 20,
  "vertical_margin": 5,
  "sift_match_thresh" : 0.6,
  "disp_range_extra_margin" : 0.2,
  "n_gcp_per_axis" : 5,
  "epipolar_thresh" : 0.5,
  "debug" : false,
  "temporary_dir" : "/tmp",
  "tile_size" : 1000,
  "clean_tmp" : true,
  "clean_intermediate" : false,
  "fusion_thresh" : 3,
  "disp_range_method" : "sift",
  "skip_existing" : false,
  "msk_erosion": 0,
  "dsm_resolution": 0.5,
  "max_processes": 16, 
  "register_with_shear": true
}

[ClementPeyrardGSI]

The ROI is valid as far as I have checked.
Can you confirm there is no need to crop a pair of images to the same extent and same resolution? Or is the ROI defined is the JSON based on the first image of the pair, and then geographically mapped to the correct location in the second / other images?

And indeed, the dataset I am using is actually from a different spacenet/DG-based competition on TopCoder (Urban 3D challenge: https://spacenetchallenge.github.io/datasets/Urban_3D_Challenge_summary.html ).
I should rename the thread, sorry for the confusion.

Have you experimented with data from this Urban3D/TopCoder challenge ?
I have tried reading RPCs directly from geotiffs, but the scripts hangs (on the elevation interpretation of RPCs I believe, see the interrupt traceback below), while it does not when reading from XML (but does not produce good results as mentioned).
So this might definitively be due to a RPC problem.

INFO: trying reading rpc from image {'rpc': '/tmp/tmpadAedW.rpc', u'img': u'/data_iarpa/Satellite-Images/UCSD/WV3/PAN/05JAN15WV031300015JAN05183041-P1BS-500647758090_01_P001_________AAE_0AAAAABPABS0.tif'}
INFO: trying reading rpc from image {'rpc': '/tmp/tmpvN0uP4.rpc', u'img': u'/data_iarpa/Satellite-Images/UCSD/WV3/PAN/08AUG15WV031300015AUG08184609-P1BS-500647758010_01_P001_________AAE_0AAAAABPABN0.tif'}
^CTraceback (most recent call last):
  File "s2p.py", line 853, in <module>
    main(user_cfg, args.step)
  File "s2p.py", line 714, in main
    initialization.build_cfg(user_cfg)
  File "/s2p/s2plib/initialization.py", line 140, in build_cfg
    cfg['utm_zone'] = rpc_utils.utm_zone(cfg['images'][0]['rpc'], x, y, w, h)
  File "/s2p/s2plib/rpc_utils.py", line 349, in utm_zone
    lon, lat = rpc.direct_estimate(x + .5*w, y + .5*h, rpc.altOff)[:2]
  File "/s2p/s2plib/rpc_model.py", line 281, in direct_estimate
    return self.direct_estimate_iterative(col, lin, alt, return_normalized)
  File "/s2p/s2plib/rpc_model.py", line 368, in direct_estimate_iterative
    y2 = apply_rfm(self.inverseLinNum, self.inverseLinDen, lat + EPS, lon, cAlt)
  File "/s2p/s2plib/rpc_model.py", line 50, in apply_rfm
    return apply_poly(num, x, y, z) / apply_poly(den, x, y, z)
  File "/s2p/s2plib/rpc_model.py", line 31, in apply_poly
    out += poly[12]*y*x*x + poly[13]*y*z*z + poly[14]*y*y*x
KeyboardInterrupt

I also tried on fresh data from DG/WV03, and everything works fine, except I get negative elevation (I'll check the other issues about that or can raise a new one if useful ?).

I will give a try to the IARPA challenge dataset from local files, but your example works perfectly, thanks.

[gfacciol]

I don't know why but the TIFs and lvl1 TIFs don't contain valid localization information.
The NTF files do, and work perfectly as seen below

  "images" : [
    { "img" : "05JAN15WV031300015JAN05183041-P1BS-500647758090_01_P001_________AAE_0AAAAABPABS0.NTF"},
    { "img" : "08AUG15WV031300015AUG08184609-P1BS-500647758010_01_P001_________AAE_0AAAAABPABN0.NTF"}
    ],

[ClementPeyrardGSI]

Wow, NTF it was, works for me as well. Thanks.