Heatmaps are patchy and not smooth

[Rohith Sai Reddy]

I have been using the tif images to create the heatmaps for my images, Below is the configuration that I am using.

exp_arguments:
  # number of classes
  n_classes: 2
  # name tag for saving generated figures and assets
  save_exp_code: HEATMAP_OUTPUT 
  # where to save raw asset files
  raw_save_dir: heatmaps/heatmap_raw_results
  # where to save final heatmaps
  production_save_dir: heatmaps/heatmap_production_results
  batch_size: 256
data_arguments: 
  # where is data stored; can be a single str path or a dictionary of key, data_dir mapping
  data_dir: heatmaps/demo/slides/
  # column name for key in data_dir (if a dict mapping is used)
  data_dir_key: source
  # csv list containing slide_ids (can additionally have seg/patch paramters, class labels, etc.)
  process_list: heatmap_demo_dataset.csv
  # preset file for segmentation/patching
  preset: presets/bwh_biopsy.csv
  # file extention for slides
  slide_ext: .tif
  # label dictionary for str: interger mapping (optional)
  label_dict:
    LUAD: 0
    LSCC: 1                        
patching_arguments:
  # arguments for patching
  patch_size: 1
  overlap: 0.1
  patch_level: 0
  custom_downsample: 1
encoder_arguments:
  # arguments for the pretrained encoder model
  model_name: resnet50_trunc # currently support: resnet50_trunc, uni_v1, conch_v1
  target_img_size: 224 # resize images to this size before feeding to encoder
model_arguments: 
  # arguments for initializing model from checkpoint
  ckpt_path: heatmaps/demo/ckpts/s_0_checkpoint.pt
  model_type: clam_sb # see utils/eval_utils/
  initiate_fn: initiate_model # see utils/eval_utils/
  model_size: small
  drop_out: 0.
  embed_dim: 1024
heatmap_arguments:
  # downsample at which to visualize heatmap (-1 refers to downsample closest to 32x downsample)
  vis_level: 1
  # transparency for overlaying heatmap on background (0: background only, 1: foreground only)
  alpha: 0.4
  # whether to use a blank canvas instead of original slide
  blank_canvas: false
  # whether to also save the original H&E image
  save_orig: true
  # file extension for saving heatmap/original image
  save_ext: jpg
  # whether to calculate percentile scores in reference to the set of non-overlapping patches
  use_ref_scores: true
  # whether to use gaussian blur for further smoothing
  blur: false
  # whether to shift the 4 default corner points for checking if a patch is inside a foreground contour
  use_center_shift: true
  # whether to only compute heatmap for ROI specified by x1, x2, y1, y2
  use_roi: false 
  # whether to calculate heatmap with specified overlap (by default, coarse heatmap without overlap is always calculated)
  calc_heatmap: true
  # whether to binarize attention scores
  binarize: false
  # binarization threshold: (0, 1)
  binary_thresh: -1
  # factor for downscaling the heatmap before final dispaly
  custom_downsample: 1
  cmap: jet
sample_arguments:
  samples:
    - name: "topk_high_attention"
      sample: true
      seed: 1
      k: 15 # save top-k patches
      mode: topk

And here I tried to change the following settings majorly - patch_size, overlap, patch_level, blur.

But still the heatmap output is like patches insted of the smooth heatmaps.

Any help and insights on how to get a smooth heatmap would be greatly appriciated.

Thanks.

[Pranav Swaroop Gundla]

@rohith2011 I had the same issue before during my work, try to change the blur: true in your config it will make further smoothing to your output. If possible increase your overlap size, the optimal would be ranging from 0.3 - 0.5. And also for futher info look into this Supplementary Figure 4. Attention heatmap visualization using varying degrees of overlap.This would do the job perfectly.