Cannot change swatch colors using ggthemr()

[Elbert]

Each time I use "ggthemr" to change my theme, I have to change the colors manually. It seems that function ggthemr() does nothing to colors.
Here are my codes as an example.

library(ggplot2)
library(ggthemr)
ggthemr("chalk", "scientific")
ggplot(mpg, aes(displ, hwy, color = factor(cyl), shape = factor(cyl))) +
    geom_jitter() +
    geom_smooth(method = lm, formula = y ~ poly(x, 2)) +
    labs(shape = "cyl", color = "cyl")

Generates:

library(ggplot2)
library(ggthemr)
ggthemr("chalk", "scientific")
defaultColor <- swatch()[seq_len(swatch())]
ggplot(mpg, aes(displ, hwy, color = factor(cyl), shape = factor(cyl))) +
    scale_color_manual(values = defaultColor[2:5]) +
    geom_jitter() +
    geom_smooth(method = lm, formula = y ~ poly(x, 2)) +
    labs(shape = "cyl", color = "cyl")

Generates: