Is there anyway to get node html when context deadline occurred?
lilixiong2018 opened this issue · comments
When I open a page with chromedp and it happend that context deadline occurred, which the main content of page are loaded finish and the node what I want are complete visible and can be visit by document.querySelectorAll. In this candition, is there anyway to capture the outer html of the given node?
For example, when context deadline occurred (random and often), the chromedp.OuterHTML(div.left_section, &html) return html with empty data, but document.querySelectorAll() on console return success.
package main
import (
"context"
"log"
"time"
"os"
"github.com/chromedp/chromedp"
)
func main() {
ctx, _ := chromedp.NewExecAllocator(
context.Background(),
append(
chromedp.DefaultExecAllocatorOptions[:],
chromedp.Flag("headless", false),
chromedp.UserAgent(`Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36`),
)...,
)
ctx, _ = context.WithTimeout(ctx, 120*time.Second)
ctx, _ = chromedp.NewContext(
ctx,
chromedp.WithDebugf(log.Printf),
)
defer chromedp.Cancel(ctx)
html := ""
chromedp.Run(ctx,
chromedp.Navigate("https://tieba.baidu.com/p/7191142093"),
chromedp.Sleep(60 * time.Second),
chromedp.WaitVisible(`div.left_section`),
chromedp.OuterHTML(`div.left_section`, &html),
)
os.WriteFile("a.html", []byte(html), 0666)
}