Skip to content


明暗对照:面向真实世界图像的光照感知矢量化方法 ​

Abstract ​

Image vectorization aims to convert raster images into editable, scalable vector representations while preserving visual fidelity. Existing vectorization methods struggle to represent complex real-world images, often producing fragmented shapes at the cost of semantic conciseness. In this paper, we propose COVec, an illumination-aware vectorization framework inspired by the Clair-Obscur principle of light–shade contrast. COVec is the first to introduce intrinsic image decomposition in the vector domain, separating an image into albedo, shade, and light layers in a unified vector representation. A semantic-guided initialization and two-stage optimization refine these layers with differentiable rendering. Experiments on various datasets demonstrate that COVec achieves higher visual fidelity and significantly improved editability compared to existing methods. The code will be released at https://github.com/decade-de/COVec.

图像矢量化旨在将栅格图像转换为可编辑、可缩放的矢量表示,同时保持视觉保真度。 现有矢量化方法难以表示复杂的真实世界图像,往往会生成碎片化的形状,并以牺牲语义简洁性为代价。 本文提出 COVec,这是一种受明暗对照(Clair-Obscur)艺术原理启发的光照感知矢量化框架。 COVec 首次将本征图像分解引入矢量域,在统一的矢量表示中,将图像分解为反照率层、阴影层和光照层。 通过语义引导的初始化和两阶段优化,借助可微渲染对这些图层进行细化。 在多个数据集上的实验表明,与现有方法相比,COVec 实现了更高的视觉保真度,并显著提升了可编辑性。 代码将发布于 https://github.com/decade-de/COVec。

1. Introduction ​

Scalable Vector Graphics (SVG) represent images through geometric primitives, providing a compact and structured description of visual content. Transform raster images into vector graphics, namely image vectorization, enables scalable, resolution-independent representations that support downstream tasks such as editing, animation, compression, and high-quality rendering across diverse display settings.

可缩放矢量图形(SVG)通过几何图元表示图像,为视觉内容提供紧凑且结构化的描述。 将栅格图像转换为矢量图形,即图像矢量化,能够得到可缩放、与分辨率无关的表示,从而支持编辑、动画、压缩,以及不同显示环境下的高质量渲染等下游任务。

While existing vectorization methods perform well on simple images such as icons and emojis, extending them to real-world photographs remains challenging. To match complex tonal variations, current methods often produce numerous fragmented shapes. This fragmentation undermines semantic conciseness— the ability to represent an image using a small number of coherent, semantically meaningful vector shapes, which in turn limits the editability of the resulting SVG.

尽管现有矢量化方法在图标和表情符号等简单图像上表现良好,但将其扩展到真实世界照片仍然面临挑战。 为了匹配复杂的色调变化,现有方法通常会生成大量碎片化的形状。 这种碎片化破坏了语义简洁性,即用少量连贯且具有明确语义的矢量形状表示图像的能力,进而限制了生成 SVG 的可编辑性。

我们的方法的分层渲染与编辑结果。上方各行展示反照率层、阴影层和光照层的逐步合成过程,说明照明如何增强纵深感与真实感。下方一行展示图层级可编辑性:仅修改反照率层并保持照明不变,即可灵活编辑,而不破坏原始明暗结构。
图1:我们的方法的分层渲染与编辑结果。上方各行展示反照率层、阴影层和光照层的逐步合成过程,说明照明如何增强纵深感与真实感。下方一行展示图层级可编辑性:仅修改反照率层并保持照明不变,即可灵活编辑,而不破坏原始明暗结构。
一幅图像的反照率层、阴影层和光照层。
图2:一幅图像的反照率层、阴影层和光照层。

In this paper, we focus on real-world image vectorization, aiming at preserving both visual fidelity and semantic conciseness. Unlike icons or stylized illustrations with uniform tones, real photographs contain rich tonal variation induced by lighting and shadow. Artists traditionally address this complexity through Clair-Obscur, an illumination-driven abstraction which convey form and depth by contrasting light and shade, as shown in Figure 3. Drawing inspiration from this principle and recent advances in intrinsic image decomposition, we decompose images into three semantically meaningful vector layers: albedo, shade and light layers. As shown in Figure 2, the albedo layer represents intrinsic surface color independent of illumination, the shade layer captures light attenuation, and the light layer models additive highlights and reflections. This decomposition enables explicit separation of albedo and illumination, producing representations that remain both semantically concise and visually consistent.

本文聚焦真实世界图像的矢量化,旨在同时保持视觉保真度与语义简洁性。 与色调均匀的图标或风格化插画不同,真实照片包含由光照和阴影引起的丰富色调变化。 艺术家通常通过 Clair-Obscur(明暗对照)来处理这种复杂性;这是一种由光照驱动的抽象手法,通过明暗对比传达形体与纵深,如 图3 所示。 受这一原理以及本征图像分解最新进展的启发,我们将图像分解为三个具有明确语义的矢量图层:反照率层、阴影层和光照层。 如 图2 所示,反照率层 表示独立于光照的表面本征颜色,阴影层 刻画光的衰减,而 光照层 对加性高光和反射进行建模。 这种分解显式分离了反照率与照明,从而得到既语义简洁又视觉一致的表示。

艺术中的明暗对照(Clair-Obscur)原理。古典绘画和现代动画利用同一语义区域(如皮肤、头发)内的色调变化来表现明暗结构。
图3:艺术中的明暗对照(Clair-Obscur)原理。古典绘画和现代动画利用同一语义区域(如皮肤、头发)内的色调变化来表现明暗结构。

Building upon this layered formulation inspired by Clair-Obscur, we introduce COVec, an illumination-aware method for real-world image Vectorization. COVec decomposes an image into albedo, shade, and light layers within a unified SVG representation. Specifically, to ensure semantically concise layering, we propose a region-wise semantic binarization strategy to initialize the albedo and illumination layers. Next, both layers are jointly optimized via differentiable rendering, enabling consistent reconstruction of color and shading. Subsequently, the illumination layer is refined individually to capture fine-grained tonal variations and then separated into shade and light components according to highlight localization. The process ultimately yields a unified SVG representation integrating albedo, shade, and light layers with native SVG blending.

基于这种受 Clair-Obscur 启发的分层建模,我们提出 COVec,一种面向真实世界图像的光照感知矢量化(Vectorization)方法。 COVec 在统一的 SVG 表示中,将图像分解为反照率层、阴影层和光照层。 具体而言,为确保分层在语义上的简洁性,我们提出区域级语义二值化策略,用于初始化反照率层与照明层。 随后,通过可微渲染联合优化这两个图层,实现颜色与阴影的一致重建。 接着,单独细化照明层以捕捉细粒度的色调变化,再依据高光位置将其分离为阴影分量与光照分量。 最终,利用 SVG 原生混合方式,得到集成反照率层、阴影层和光照层的统一 SVG 表示。

Extensive experiments on various real-world datasets validate the effectiveness of COVec (Figure 1). Our method performs well not only on complex real-world images but also on simpler datasets such as emojis. COVec achieves lower reconstruction and perceptual errors under different path constraints, demonstrating its robustness and visual fidelity. The semantically concise decomposition enables intuitive and illumination-consistent editing: recoloring the albedo layer naturally adjusts appearance without disrupting illumination structure. Compared with existing methods, COVec attains visually coherent results with significantly fewer path edits. Ablation studies demonstrate the superiority of our proposed region-wise semantic binarization strategy for illumination initialization.

在多个真实世界数据集上的大量实验验证了 COVec 的有效性(图1)。 我们的方法不仅在复杂的真实世界图像上表现良好,也适用于表情符号等更简单的数据集。 在不同的路径数量约束下,COVec 均获得了更低的重建误差和感知误差,体现出其鲁棒性与视觉保真度。 这种语义简洁的分解支持直观且保持光照一致的编辑:对反照率层重新着色,可以自然调整外观,而不会破坏照明结构。 与现有方法相比,COVec 只需显著更少的路径编辑,就能获得视觉连贯的结果。 消融研究表明,我们提出的区域级语义二值化策略在照明初始化方面具有优势。

To summarize, our contributions are as follows:

总而言之,我们的贡献如下:

  • We propose COVec, an illumination-aware vectorization method that, for the first time, leverages intrinsic decomposition into image vectorization, enabling consistent and concise reconstruction for real-world images.
  • We design a region-wise semantic binarization strategy to initialize vector layers and demonstrate its effectiveness for illumination modeling.
  • Compared with previous methods, COVec achieves higher visual fidelity and also semantically concise vector representations, resulting in enhanced editability and structural coherence.
  • 我们提出 COVec,一种光照感知矢量化方法,首次将本征分解引入图像矢量化,从而对真实世界图像进行一致且简洁的重建。
  • 我们设计了一种区域级语义二值化策略来初始化矢量图层,并证明了其在照明建模中的有效性。
  • 与以往方法相比,COVec 实现了更高的视觉保真度,同时获得了语义简洁的矢量表示,从而增强了可编辑性与结构连贯性。

2. Related Work ​

2.1. Image Vectorization ​

Image vectorization converts raster images into vector representations composed of geometric primitives such as polygons and Bézier curves. Early approaches segment regions and approximate their color or contour boundaries with vector shapes, while later works employ more expressive primitives, such as diffusion curves and gradient meshes, to capture smooth color transitions.

图像矢量化将栅格图像转换为由多边形、Bézier 曲线等几何图元构成的矢量表示。 早期方法先分割区域,再用矢量形状逼近其颜色边界或轮廓边界;后续工作则采用扩散曲线、梯度网格等更具表达能力的图元,以捕捉平滑的颜色过渡。

Recent advances in deep learning have brought significant progress to this field. These methods for generating vector graphics can be broadly categorized into optimization-based and dataset-based approaches. Dataset-based methods rely on large-scale datasets to learn vector generation models. Early works such as SVG-VAE and DeepSVG learn latent representations of simple vector icons, while more recent large-scale models like StarVector attempt to extend this paradigm using transformer architectures.

深度学习的最新进展推动了这一领域的显著发展。 这些矢量图形生成方法大致可分为基于优化的方法和基于数据集的方法。 基于数据集的方法依赖大规模数据集来学习矢量生成模型。 SVG-VAE 和 DeepSVG 等早期工作学习简单矢量图标的潜在表示,而 StarVector 等近期大规模模型则尝试利用 Transformer 架构扩展这一范式。

However, due to data scarcity and domain bias, these dataset-based models generalize poorly to real-world images, often producing incomplete or empty SVGs. In contrast, optimization-based methods directly fit geometric primitives to an input image without requiring training data. They are highly generalizable and can be applied across diverse visual domains. The differentiable rasterizer DiffVG has greatly advanced this direction, enabling gradient-based optimization of vector primitives and inspiring subsequent works such as LIVE and LayerVec.

然而,由于数据稀缺和领域偏差,这些基于数据集的模型对真实世界图像的泛化能力较差,往往生成不完整甚至空白的 SVG。 相比之下,基于优化的方法直接用几何图元拟合输入图像,无需训练数据。 这类方法具有较强的泛化能力,可应用于不同的视觉领域。 可微栅格化器 DiffVG 极大推动了这一方向的发展,使基于梯度的矢量图元优化成为可能,并启发了 LIVE、LayerVec 等后续工作。

To further enhance structural organization and editability, some optimization-based works explore layer decomposition, where an image is represented as multiple vector layers for hierarchical editing. LIVE introduces a bottom-up SVG construction framework that progressively adds and optimizes Bézier paths using differentiable rendering. O&R adopts an opposite, top-down strategy, iteratively optimizing and pruning Bézier curves according to their contribution to the overall reconstruction quality. LayerVec employs a semantically guided, progressive vectorization pipeline. It reconstructs images from coarse to fine guided by a sequence of simplified images.

为进一步增强结构组织与可编辑性,一些基于优化的工作探索了图层分解,将图像表示为多个矢量图层,以支持层级编辑。 LIVE 提出一种自底向上的 SVG 构建框架,利用可微渲染逐步添加并优化 Bézier 路径。 O&R 采用相反的自顶向下策略,根据 Bézier 曲线对整体重建质量的贡献,迭代优化并裁剪曲线。 LayerVec 采用语义引导的渐进式矢量化流程。 它在一系列简化图像的引导下,由粗到细地重建图像。

However, existing layer decomposition techniques mainly operate within RGBA compositing frameworks. When applied to real-world images, they often produce redundant vector paths that compromise semantic conciseness. In contrast, our method leverages the rich blending modes supported by the SVG format, such as multiplicative blending to simulate realistic shading effects, to explicitly decouple albedo, shade, and light layers, achieving semantically meaningful and editable vector representations.

然而,现有图层分解技术主要在 RGBA 合成框架下运行。 应用于真实世界图像时,这些方法往往生成冗余矢量路径,损害语义简洁性。 相比之下,我们的方法利用 SVG 格式支持的丰富混合模式,例如通过乘法混合模拟真实的阴影效果,显式解耦反照率层、阴影层和光照层,从而获得语义明确且可编辑的矢量表示。

2.2. Intrinsic Image Decomposition ​

Intrinsic image decomposition aims to separate an image into its albedo and illumination components, enabling illumination-aware manipulation. Classical intrinsic models adopt the Lambertian diffuse assumption, expressing an image as a multiplicative combination of albedo and grayscale shading. These models are effective for simple scenes but fail to capture colored lighting and specular effects, leading to inaccurate albedo estimation in complex, real-world environments. To overcome the limitations of grayscale shading, recent works have developed the model to incorporate RGB shading, accounting for colored illumination. Careaga and Aksoy adopt an extended image formation model that decomposes images into albedo, colorful shading, and a residual component for specularities and light sources. By relaxing the Lambertian and single-color assumptions, it achieves illumination-aware decomposition for in-the-wild photographs.

本征图像分解旨在将图像分离为反照率与照明分量,以支持具有光照感知能力的操作。 经典本征模型采用朗伯漫反射假设,将图像表示为反照率与灰度阴影的乘法组合。 这些模型适用于简单场景,但无法捕捉彩色光照和镜面反射效应,导致在复杂的真实世界环境中对反照率的估计不准确。 为克服灰度阴影的局限,近期工作将模型扩展为包含 RGB 阴影,以考虑彩色照明。 Careaga 和 Aksoy 采用了一种扩展的图像形成模型,将图像分解为反照率、彩色阴影,以及用于镜面反射和光源的残差分量。 通过放宽朗伯假设与单色假设,该模型实现了对自然场景照片的光照感知分解。

However, these methods operate purely in the pixel domain, where decomposed components are produced as separate raster images. These images must be manually combined in professional editing software to achieve a coherent rendering result. In contrast, we represent albedo, shade, and light as distinct vector layers within a unified SVG output, leveraging SVG’s native blending modes for compositing and editing.

然而,这些方法完全在像素域运行,分解后的分量以独立栅格图像的形式输出。 这些图像必须在专业编辑软件中手动合成,才能获得协调的渲染结果。 相比之下,我们在统一的 SVG 输出中,将反照率、阴影和光照表示为不同的矢量图层,利用 SVG 原生混合模式实现合成与编辑。

**COVec 的流程。** 我们采用不同策略初始化反照率层与照明层,获得各自的掩码。第一阶段通过可微渲染联合优化两个图层,其中 $\mathcal{L}_{\text{struct}}$ 保证与分割掩码的结构一致性,$\mathcal{L}_{\text{recon}}$ 最小化混合结果([式(4)](#equation-4))与目标图像之间的重建误差。第二阶段通过 $\mathcal{L}_{\text{refine}}$ 细化照明细节。最后,将照明层分离为阴影和光照分量,生成统一的 SVG 输出。

图4:COVec 的流程。 我们采用不同策略初始化反照率层与照明层,获得各自的掩码。第一阶段通过可微渲染联合优化两个图层,其中 Lstruct 保证与分割掩码的结构一致性,Lrecon 最小化混合结果(式(4))与目标图像之间的重建误差。第二阶段通过 Lrefine 细化照明细节。最后,将照明层分离为阴影和光照分量,生成统一的 SVG 输出。

图4:COVec 的流程。 我们采用不同策略初始化反照率层与照明层,获得各自的掩码。第一阶段通过可微渲染联合优化两个图层,其中 Lstruct 保证与分割掩码的结构一致性,Lrecon 最小化混合结果(式(4))与目标图像之间的重建误差。第二阶段通过 Lrefine 细化照明细节。最后,将照明层分离为阴影和光照分量,生成统一的 SVG 输出。

3. Preliminary: Light–Shade Decomposition ​

Following the extended image formation model adopted by [3, 33, 41], a pixel image I can be represented as the combination of intrinsic albedo, shading, and additive lighting:

根据文献 [3、33、41] 采用的扩展图像形成模型,像素图像 I 可表示为本征反照率、阴影与加性光照的组合:

(1)I=A∗S+L,

where A,S,L,I∈[0,1]H×W×3 denote per-pixel RGB albedo, shading, lighting, and observed image, respectively. We adapt this principle to image vectorization. Specifically, the target SVG representation Vfinal can be formulated as the composition of three vector layers:

其中,A,S,L,I∈[0,1]H×W×3 分别表示逐像素的 RGB 反照率、阴影、光照和观测图像。 我们将这一原理应用于图像矢量化。 具体而言,目标 SVG 表示 Vfinal 可写成三个矢量图层的组合:

(2)Vfinal=VA∗VS+VL,

where VA, VS, and VL denote the albedo, shade, and light layers, which are interpreted as follows:

其中,VA、VS 和 VL 分别表示反照率层、阴影层和光照层,其含义如下:

  • The albedo layer (VA) represents the intrinsic surface reflectance, capturing colors that remain constant under varying illumination.
  • The shade layer (VS) models geometric light attenuation caused by visibility and orientation, darkening regions in proportion to shading intensity. It is composited with the albedo layer using the multiply blending mode.
  • The light layer (VL) accounts for additive illumination effects such as highlights, reflections, and scattered light, composited through the plus-lighter blending mode.
  • 反照率层(VA)表示表面的本征反射率,捕捉在光照变化下保持不变的颜色。
  • 阴影层(VS)对由可见性和朝向引起的几何性光衰减进行建模,按阴影强度使相应区域变暗。它通过 multiply 混合模式与反照率层合成。
  • 光照层(VL)刻画高光、反射和散射光等加性照明效应,并通过 plus-lighter 混合模式进行合成。

Each decomposition layer V∗ is represented as a collection of parameterized vector paths:

每个分解图层 V∗ 都表示为一组参数化的矢量路径:

(3)V∗=∑n=1N∗θn∗,θn∗={Pn,Cn,τn},

where a vector path is defined by its geometric shapes Pn, fill color Cn, and opacity τn∈[0,1].

其中,一条矢量路径由其几何形状 Pn、填充颜色 Cn 和不透明度 τn∈[0,1] 定义。

Practically, directly optimizing the three components in Eq. (2) is challenging due to the strong coupling among variables. To simplify the optimization, we draw upon the simplified Lambertian intrinsic decomposition, where image intensities are modeled as the product of albedo and illumination. We first decompose the objective into two vector layers: the albedo layer and illumination layer, and jointly optimize them:

在实践中,由于变量之间存在强耦合,直接优化 式(2) 中的三个分量较为困难。 为简化优化过程,我们借鉴简化的朗伯本征分解,将图像强度建模为反照率与照明的乘积。 我们首先将目标分解为反照率层与照明层这两个矢量图层,并对它们进行联合优化:

(4)Vfinal=VA∗VI,

where VI denotes the illumination layer which combines shading and lighting components. Afterward, we refine it with VA fixed and then separate it into shade and light layer.

其中,VI 表示融合了阴影与光照分量的照明层。 随后,在固定 VA 的情况下对其细化,再将其分离为阴影层与光照层。

4. Method ​

4.1. Overview of Pipeline ​

As shown in Figure 4, the overall COVec pipeline follows a coarse-to-fine reconstruction process, including initialization and optimization. We begin with the initialization of VA and VI. To ensure clear layer separation, we adopt distinct initialization strategies for the two decomposition layers. The albedo layer VA is initialized from illumination-invariant albedo masks extracted by a pretrained segmentation model. In contrast, the illumination layer VI is initialized through a region-wise semantic binarization on the target image.

如 图4 所示,COVec 的整体流程遵循由粗到细的重建过程,包括初始化和优化。 我们首先初始化 VA 与 VI。 为确保清晰的图层分离,我们对这两个分解图层采用不同的初始化策略。 反照率层 VA 由预训练分割模型提取的、对光照变化保持不变的反照率掩码初始化。 相比之下,照明层 VI 通过对目标图像执行区域级语义二值化来初始化。

In the first stage of optimization, we jointly optimize VA and VI via differentiable rendering. The objective contains a structure loss computed with respect to each layer’s mask and a reconstruction loss between the blended composite and the target image. Subsequently, in the second stage, VA is fixed, and the illumination layer VI is refined by introducing new vector paths and minimizing a refinement loss to capture fine-grained lighting variations. Finally, we separate the shade and light layers, VS and VL, from the illumination layer VI. The resulting three decomposition layers are then composited according to the image formation model described in Eq. (2), using SVG blending modes to produce a unified and editable vector output.

在优化的第一阶段,我们通过可微渲染联合优化 VA 与 VI。 目标函数包含针对各图层掩码计算的结构损失,以及混合合成结果与目标图像之间的重建损失。 随后,在第二阶段固定 VA,通过引入新的矢量路径并最小化细化损失来细化照明层 VI,以捕捉细粒度的光照变化。 最后,我们从照明层 VI 中分离出阴影层 VS 和光照层 VL。 随后,按照 式(2) 描述的图像形成模型,利用 SVG 混合模式合成这三个分解图层,生成统一且可编辑的矢量输出。

4.2. Layer Initialization ​

A good initialization is crucial for efficient optimization, as it provides a reliable starting direction and significantly reduces optimization time. To enable vector-based optimization, we initialize the albedo and illumination layers as explicit regions.

良好的初始化对于高效优化至关重要,因为它能够提供可靠的起始方向,并显著减少优化时间。 为实现基于矢量的优化,我们将反照率层与照明层初始化为显式区域。

Albedo Masks. The albedo layer represents the intrinsic reflectance of surfaces. We first obtain the albedo map from the target image using a standard intrinsic decomposition model. However, directly using the raw albedo intensity does not provide clear structural regions. To extract semantic structural regions that are unaffected by illumination variations, we employ a pretrained segmentation model, SAM, on the albedo map to generate albedo masks. These masks define the vector regions for VA.

反照率掩码。 反照率层表示表面的本征反射率。 我们首先使用标准的本征分解模型,从目标图像中获得反照率图。 然而,直接使用原始反照率强度无法得到清晰的结构区域。 为提取不受光照变化影响的语义结构区域,我们在反照率图上应用预训练分割模型 SAM,生成反照率掩码。 这些掩码定义了 VA 的矢量区域。

Illumination Masks. The illumination layer provides an estimate of the illumination components, including shading and lighting. At the initialization stage, the illumination layer is approximated by extracting shadow regions from the target image. We observe that the intensity of shadows varies across different regions of an image. Therefore, we propose a region-wise semantic binarization strategy to adaptively capture such spatial variations. As shown in Figure 5, instead of thresholding the entire image uniformly, we perform binarization independently within each semantic region of the target image. Pixels outside these semantic regions are ignored, ensuring that binarization operates only within meaningful object regions.

照明掩码。 照明层提供对包括阴影与光照在内的照明分量的估计。 在初始化阶段,通过从目标图像中提取阴影区域来近似照明层。 我们观察到,图像不同区域的阴影强度有所不同。 因此,我们提出 区域级语义二值化策略,以自适应地捕捉这种空间变化。 如 图5 所示,我们并非对整幅图像统一设置阈值,而是在目标图像的每个语义区域内独立进行二值化。 这些语义区域之外的像素会被忽略,从而确保二值化仅作用于有意义的对象区域。

Let {m1A,m2A,…,mnA}⊂Rw×h denote the set of albedo masks corresponding to different semantic regions in the target image. For each albedo mask miA, a region-wise threshold is computed as the mean pixel intensity within that region:

设 {m1A,m2A,…,mnA}⊂Rw×h 表示与目标图像中不同语义区域对应的反照率掩码集合。 对于每个反照率掩码 miA,将该区域内像素强度的均值作为区域级阈值:

(5)Ti=1|miA|∑p∈miAI(p),

where I(p) denotes the pixel intensity of the target image I at pixel p. Pixels of the target image that fall within miA and satisfy I(p)≤Ti are classified as shadow pixels, forming illumination masks. These masks define the vector regions for VI.

其中,I(p) 表示目标图像 I 在像素 p 处的像素强度。 目标图像中落在 miA 内且满足 I(p)≤Ti 的像素被归为阴影像素,由此形成照明掩码。 这些掩码定义了 VI 的矢量区域。

区域级语义二值化过程示意。在每个语义掩码内对目标图像进行二值化,得到与对象边界对齐的照明掩码。
图5:区域级语义二值化过程示意。在每个语义掩码内对目标图像进行二值化,得到与对象边界对齐的照明掩码。

Organization of Segmented Masks. After obtaining the masks of the two layers, we organize them into hierarchical mask groups following a back-to-front ordering strategy from LayerVec. Specifically, mask organization is performed independently within each layer. Larger masks are placed in bottom groups, while smaller ones are placed on top to avoid overlap within a single group. Each mask boundary is extracted and simplified using the Douglas–Peucker algorithm to reduce redundant points while preserving structural fidelity. For each mask, the corresponding vector shape is represented as a closed cubic Bézier contour, with control points sampled from the simplified boundary.

分割掩码的组织。 获得两个图层的掩码后,我们采用 LayerVec 的从后向前排序策略,将其组织为层次化的掩码组。 具体而言,各图层内的掩码分别独立组织。 较大的掩码放置在底层组中,较小的掩码放在上层,以避免同一组内出现重叠。 我们提取每个掩码的边界,并使用 Douglas–Peucker 算法进行简化,在保持结构保真度的同时减少冗余点。 对于每个掩码,其对应的矢量形状表示为闭合的三次 Bézier 轮廓,控制点从简化后的边界中采样得到。

After the initialization and organization of segmented masks, both the albedo and illumination layers consist of multiple vector path groups organized hierarchically.

完成分割掩码的初始化与组织后,反照率层和照明层均由多个按层次组织的矢量路径组构成。

4.3. Structural Optimization ​

To effectively optimize the two-layer vector decomposition, we adopt a progressive strategy that first stabilizes the geometric structure of each layer and then refines their joint blended result.

为有效优化双层矢量分解,我们采用渐进策略:先稳定各图层的几何结构,再细化它们的联合混合结果。

Structure Loss. We adopt a layer-wise structure loss to ensure geometric consistency between the vectorized representation and its mask group. For each path group Ijgroup and its mask group Ijmask, the structure loss is formulated as:

结构损失。 我们采用逐层结构损失,确保矢量化表示与其掩码组之间的几何一致性。 对于每个路径组 Ijgroup 及其掩码组 Ijmask,结构损失定义为:

(6)Lstruct=∑j=1N(‖Ijmask−Ijgroup‖22+λ∑p∈Igroup,j′ReLU(δ−α(p))),

where N is the number of path groups, vectors are rendered with the same semi-transparent gray in Igroup,j′, α(p) is the pixel transparency of pixel p, δ is the threshold controlling overlap tolerance, and λ is a weighting factor set to 1×10−8. The first term enforces shape alignment between the vector paths and reference masks with pixel-wise MSE loss, while the second term discourages excessive self-overlap within the same group. This structure loss guides each path group toward compact and non-redundant geometric representations.

其中,N 是路径组的数量,矢量在 Igroup,j′ 中以相同的半透明灰色渲染,α(p) 是像素 p 的透明度,δ 是控制重叠容忍度的阈值,而 λ 是设为 1×10−8 的权重系数。 第一项通过逐像素 MSE 损失,使矢量路径与参考掩码在形状上对齐;第二项则抑制同一组内过度的自重叠。 该结构损失引导各路径组形成紧凑且无冗余的几何表示。

Reconstruction Loss. To encourage overall visual consistency, we define a shared reconstruction loss, which measures the difference between the rendered composite and the target image:

重建损失。 为促进整体视觉一致性,我们定义一个共享的重建损失,衡量渲染合成结果与目标图像之间的差异:

(7)Lrecon=‖I−R(VA)∗R(VI)‖22,

where R(⋅) denotes the differentiable rendering operator.

其中,R(⋅) 表示可微渲染算子。

Optimization Schedule. We optimize the albedo layer VA and the illumination layer VI using two independent optimizers. Optimization follows a two-stage schedule. During the warm-up stage, each layer is supervised by its own structure loss,

优化安排。 我们使用两个独立的优化器,分别优化反照率层 VA 和照明层 VI。 优化按两个阶段进行。 在预热阶段,每个图层都由其自身的结构损失监督,

(8)LA=LstructA,LI=LstructI,

ensuring stable alignment with their initialization masks, guided by the albedo masks and illumination masks respectively. After the first 50 epochs, we switch to the shared reconstruction objective for the remaining 50 epochs,

分别在反照率掩码和照明掩码的引导下,确保各图层与其初始化掩码稳定对齐。 在前 50 个训练轮次之后,剩余 50 个轮次切换到共享的重建目标,

(9)LA=LI=Lrecon,

while still updating the parameters of VA and VI with their own optimizers. This schedule first stabilizes layer geometry, then jointly refines all layers to reproduce the target image.

同时仍使用各自的优化器更新 VA 与 VI 的参数。 这一安排先稳定图层几何结构,再联合细化所有图层,以重现目标图像。

4.4. Illumination Refinement & Layer Separation ​

Illumination Refinement. With both the albedo layer and the existing illumination base fixed, new vector paths are progressively appended to the illumination layer to capture fine-grained details. These newly introduced paths collectively form an enhanced illumination layer. Following the path addition strategy of LIVE, new paths are inserted into regions with high reconstruction error. During this refinement, only the parameters of the newly added paths are optimized, while previously optimized paths remain fixed. The optimization minimizes the reconstruction loss between the rendered composite and the target image:

照明细化。 在固定反照率层与现有照明基础部分的情况下,逐步向照明层添加新的矢量路径,以捕捉细粒度细节。 这些新引入的路径共同构成增强后的照明层。 按照 LIVE 的路径添加策略,将新路径插入重建误差较大的区域。 在细化过程中,仅优化新增路径的参数,已优化过的路径保持固定。 优化目标是最小化渲染合成结果与目标图像之间的重建损失:

(10)Lrefine=‖I−R(VA)∗R(VI)‖22,

which is the same as Eq. (7), only with the parameters of VA freezed. To maintain a compact representation, we follow LayerVec and periodically perform vector cleanup operations, such as merging and removing redundant paths.

该损失与 式(7) 相同,只是冻结了 VA 的参数。 为保持表示的紧凑性,我们沿用 LayerVec 的做法,定期执行矢量清理操作,例如合并和移除冗余路径。

Layer Separation. After illumination refinement, the illumination layer VI is separated into the shade layer VS and the light layer VL. The separation is performed by examining each vector path's fill color intensity: paths whose color values remain within the normalized range [0,1] are assigned to the shade layer VS, whereas paths whose values exceed this range are identified as highlight-contributing regions and thus grouped into the light layer VL.

图层分离。 在照明细化后,将照明层 VI 分离为阴影层 VS 和光照层 VL。 分离过程通过检查每条矢量路径的填充颜色强度来完成:颜色值处于归一化范围 [0,1] 内的路径被分配到阴影层 VS;颜色值超出该范围的路径被识别为产生高光的区域,并归入光照层 VL。

During this process, the control points of all paths remain unchanged. Paths belonging to VS directly inherit both the shapes and colors from VI, while paths in VL retain only their shapes. Because the light layer uses additive blending, their colors cannot be taken from VI; instead, they are recomputed from the residual illumination. Specifically, we compute a residual map I−R(VA)∗R(VS), and the final color of each light-layer path is obtained by averaging the pixel values of the residual map within the region covered by that path.

在这一过程中,所有路径的控制点保持不变。 属于 VS 的路径直接继承 VI 中的形状与颜色,而 VL 中的路径只保留形状。 由于光照层采用加法混合,其颜色不能直接取自 VI,而需要根据残余照明重新计算。 具体而言,我们计算残差图 I−R(VA)∗R(VS),并在每条光照层路径所覆盖的区域内,对残差图的像素值取平均,得到该路径的最终颜色。

5. Experiments ​

5.1. Experimental Setup ​

Implementation Details. We implement COVec in PyTorch and optimize it using the Adam optimizer, with a learning rate of 1.0 for control point optimization and 0.01 for color parameter optimization. All experiments are conducted on four NVIDIA L40S GPUs. Additional implementation details are provided in the supplementary material.

实现细节。 我们使用 PyTorch 实现 COVec,并采用 Adam 优化器进行优化;控制点优化的学习率为 1.0,颜色参数优化的学习率为 0.01。 所有实验均在四张 NVIDIA L40S GPU 上进行。 更多实现细节见补充材料。

Dataset. For evaluation, we collect three datasets, each containing 100 randomly selected images. The first two focus on complex real-world images, while the third targets simple images.

数据集。 为进行评估,我们收集了三个数据集,每个数据集包含 100 张随机选取的图像。 前两个数据集聚焦复杂的真实世界图像,第三个面向简单图像。

Face Dataset. This dataset is derived from the FFHQ dataset, which contains high-quality facial photographs under diverse ethnicities, genders, and ages.

人脸数据集。 该数据集取自 FFHQ 数据集,其中包含覆盖不同族裔、性别和年龄的高质量人脸照片。

Scene Dataset. We collect images from the Things dataset and TID2013 dataset. This dataset includes a wide variety of objects, animals, and natural scenes.

场景数据集。 我们从 Things 数据集和 TID2013 数据集中收集图像。 该数据集包含多种物体、动物和自然场景。

Emoji Dataset. Following O&R, we build an emoji dataset from the Noto Emoji dataset, which consists of simple icons with clean color boundaries.

表情符号数据集。 沿用 O&R 的做法,我们基于 Noto Emoji 数据集构建表情符号数据集,其中包含颜色边界清晰的简单图标。

Model Variant for simple images. Since the emoji dataset contains visually simple images without complex illumination variation, we use a simplified version of COVec that optimizes only a single albedo layer without introducing additional light or shade layers. This degenerated configuration reduces COVec to a LayerVec-like pipeline, while still incorporating our initialization strategy. The binary masks obtained from semantic segmentation are merged into the albedo initialization process.

面向简单图像的模型变体。 由于表情符号数据集中的图像在视觉上较为简单,不含复杂的照明变化,我们使用简化版 COVec,仅优化单个反照率层,而不引入额外的光照层或阴影层。 这种退化配置使 COVec 简化为类似 LayerVec 的流程,同时仍保留我们的初始化策略。 通过语义分割获得的二值掩码被并入反照率初始化过程。

Compared Methods. We compare COVec with four representative vectorization methods: DiffVG, LIVE, O&R, and LayerVec. Among them, LayerVec represents the current state-of-the-art approach for layered image vectorization. For a fair comparison, we adopt Bézier curves as the unified vector primitive and use the same learning rate settings as in COVec across all baselines.

对比方法。 我们将 COVec 与四种具有代表性的矢量化方法进行比较:DiffVG、LIVE、O&R 和 LayerVec。 其中,LayerVec 代表了当前最先进的分层图像矢量化方法。 为公平比较,我们统一采用 Bézier 曲线作为矢量图元,并为所有基线使用与 COVec 相同的学习率设置。

5.2. Main Results ​

Visual Fidelity. We evaluate COVec against the compared methods in terms of visual fidelity, measuring the differences between target images and rendered images.

视觉保真度。 我们通过衡量目标图像与渲染图像之间的差异,评估 COVec 相对于各对比方法的视觉保真度。

Qualitative Comparison. Figure 6 presents visual comparisons of reconstructed vector images under the same path budgets. In complex real-world scenes (top and middle rows), COVec is able to reconstruct complete semantic regions of the target image, such as the child’s eyes and mouth. Moreover, COVec preserves fine-grained illumination details that other methods fail to capture. While existing approaches rely on redundant paths to approximate these details, COVec uses compact and well-aligned shapes that accurately represent challenging details, such as the fur of the bear. In the simple emoji domain (bottom row), COVec produces accurate shape contours comparable to LayerVec, validating its adaptability to simplified scenarios.

定性比较。 图6 展示了在相同路径预算下,重建矢量图像的视觉对比。 在复杂的真实世界场景中(上部与中部各行),COVec 能够重建目标图像中完整的语义区域,例如儿童的眼睛和嘴巴。 此外,COVec 保留了其他方法未能捕捉的细粒度照明细节。 现有方法依赖冗余路径来逼近这些细节,而 COVec 使用紧凑且对齐良好的形状,准确表示熊的毛发等难以处理的细节。 在简单的表情符号领域(底部一行),COVec 生成了与 LayerVec 相当的精确形状轮廓,验证了其对简化场景的适应能力。

不同数据集上的定性重建比较。人脸与场景数据集的所有示例均使用 64 条路径进行矢量化,表情符号示例则使用 16 条路径。与现有方法相比,COVec 实现了更高的视觉保真度和紧凑的矢量结构。
图6:不同数据集上的定性重建比较。人脸与场景数据集的所有示例均使用 64 条路径进行矢量化,表情符号示例则使用 16 条路径。与现有方法相比,COVec 实现了更高的视觉保真度和紧凑的矢量结构。
在不同路径数量下,以 MSE 和 LPIPS 为指标进行定量比较。
图7:在不同路径数量下,以 MSE 和 LPIPS 为指标进行定量比较。

Quantitative Comparison. Figure 7 reports both Mean Squared Error (MSE) and Learned Perceptual Image Patch Similarity (LPIPS) curves under increasing path numbers across all datasets. In most cases, COVec achieves the lowest reconstruction error and perceptual distance, consistently outperforming other methods under different numbers of paths. From complex real-world scenes to simple emoji graphics, COVec attains higher fidelity than LayerVec and O&R, achieving comparable or better results with substantially fewer paths.

定量比较。 图7 给出了在所有数据集上,随着路径数量增加,均方误差(MSE)和学习感知图像块相似度(LPIPS)的变化曲线。 在大多数情况下,COVec 获得了最低的重建误差与感知距离,并在不同路径数量下持续优于其他方法。 从复杂的真实世界场景到简单的表情符号图形,COVec 的保真度均优于 LayerVec 和 O&R,能够用显著更少的路径获得相当或更好的结果。

图层分解与编辑。COVec 将图像分解为反照率层、阴影层和光照层。我们对反照率层重新着色,同时保持阴影层与光照层固定。这保留了照明结构,而混合机制会使照明颜色自然适应新的反照率色调。
图8:图层分解与编辑。COVec 将图像分解为反照率层、阴影层和光照层。我们对反照率层重新着色,同时保持阴影层与光照层固定。这保留了照明结构,而混合机制会使照明颜色自然适应新的反照率色调。

Layer-wise Representation. To demonstrate the semantic conciseness and interpretability of our layered representation, we visualize the decomposed albedo, shade, and light layers in Figure 8. COVec explicitly separates intrinsic color and illumination, allowing intuitive editing at the layer level. By recoloring only the albedo layer while keeping the shade and light layers fixed, the overall illumination effects remain consistent. It enables flexible appearance modification without breaking illumination coherence. Moreover, thanks to the hierarchical structure of vector paths, the layers can be further decomposed into compact, part-level representations— each corresponding to meaningful semantic components. This concise organization enables efficient localized edits, such as recoloring or relighting specific parts.

分层表示。 为展示分层表示的语义简洁性与可解释性,我们在 图8 中可视化了分解得到的反照率层、阴影层和光照层。 COVec 显式分离本征颜色与照明,使图层级编辑更加直观。 只对反照率层重新着色,同时保持阴影层与光照层不变,即可维持整体照明效果的一致性。 这使得外观能够灵活修改,而不会破坏照明的连贯性。 此外,得益于矢量路径的层次结构,各图层还可以进一步分解为紧凑的部件级表示,每个表示都对应有意义的语义组件。 这种简洁的组织方式支持高效的局部编辑,例如对特定部件重新着色或重新打光。

A qualitative comparison in Figure 9 further shows that COVec yields cleaner structural organization across layers compared to existing methods. Previous methods rely on fragmented paths to approximate the image, where illumination is implicitly embedded within color regions. In contrast, our method constructs illumination in a progressive and disentangled manner: it first establishes intrinsic albedo to capture material identity, and then gradually adds paths for the shade and light layers. This process mirrors the Clair-Obscur painting principle, resulting in semantically concise and illumination-aware vector representations.

图9 中的定性比较进一步表明,与现有方法相比,COVec 能够在各图层之间形成更清晰的结构组织。 以往方法依赖碎片化路径来逼近图像,照明被隐式嵌入颜色区域之中。 相比之下,我们的方法以渐进且解耦的方式构建照明:先建立本征反照率以刻画材质属性,再逐步为阴影层与光照层添加路径。 这一过程呼应了明暗对照的绘画原理,得到语义简洁且具有光照感知能力的矢量表示。

分层表示的比较。各列展示路径数量逐步增加时的结果,揭示分层表示如何逐步组织形成。
图9:分层表示的比较。各列展示路径数量逐步增加时的结果,揭示分层表示如何逐步组织形成。
受控路径修改下的颜色编辑比较。各列对应逐渐增加的已编辑路径数量(1、2、4、8、16)。选择与差异区域重叠最多的前 k 条路径进行编辑,并重新着色以匹配目标图像。
图10:受控路径修改下的颜色编辑比较。各列对应逐渐增加的已编辑路径数量(1、2、4、8、16)。选择与差异区域重叠最多的前 k 条路径进行编辑,并重新着色以匹配目标图像。

Image Editing To assess the editability of the resulting SVG generated by different vectorization methods, we conduct a controlled color-editing experiment as shown in Figure 10. For each method, we progressively edit a small number of vector paths according to the color changes suggested by a reference image generated by Gemini 2.5. This process detects color-difference regions between the original and target images and adjusts the corresponding vector paths accordingly.

图像编辑 为评估不同矢量化方法所生成 SVG 的可编辑性,我们进行了受控的颜色编辑实验,如 图10 所示。 对于每种方法,我们依据 Gemini 2.5 生成的参考图像所指示的颜色变化,逐步编辑少量矢量路径。 该过程检测原始图像与目标图像之间的颜色差异区域,并据此调整相应的矢量路径。

As the number of edited paths increases, the SVG generated by COVec exhibits the most effective and natural transition towards the target image. In contrast, other methods produce redundant paths and do not decouple intrinsic color from illumination, making it difficult to achieve the target appearance by editing only a small subset of paths. Benefiting from our layered decomposition, COVec can edit intrinsic colors without disrupting the illumination structure, allowing precise edits with only a few path modifications.

随着被编辑路径数量的增加,COVec 生成的 SVG 能够最有效且自然地过渡到目标图像。 相比之下,其他方法生成冗余路径,且未将本征颜色与照明解耦,因此难以仅通过编辑少量路径来获得目标外观。 得益于分层分解,COVec 能够在不破坏照明结构的情况下编辑本征颜色,仅修改少数路径即可实现精确编辑。

5.3. Ablation Study ​

Albedo-based Initialization. A good initialization is essential for achieving stable layer separation and optimization. As shown in Figure 11, using the intrinsic albedo map to initialize the albedo layer provides a clear structural prior, allowing the vector paths to capture material colors that are invariant to illumination. Without the albedo map, the layer optimization tends to entangle intrinsic surface colors with illumination effects.

基于反照率的初始化。 良好的初始化对于实现稳定的图层分离与优化至关重要。 如 图11 所示,利用本征反照率图初始化反照率层,可以提供清晰的结构先验,使矢量路径捕捉不随光照变化的材质颜色。 若不使用反照率图,图层优化往往会将表面本征颜色与光照效应纠缠在一起。

基于反照率初始化的效果。使用反照率图进行初始化,可以实现对光照变化保持不变的矢量分割,以及更清晰的本征颜色分离。
图11:基于反照率初始化的效果。使用反照率图进行初始化,可以实现对光照变化保持不变的矢量分割,以及更清晰的本征颜色分离。

Region-wise Semantic Binarization. To obtain a structurally meaningful illumination layer, we compare four binarization strategies— adaptive, OTSU, global thresholding, and our proposed region-wise semantic binarization strategy. As shown in Figure 12, traditional pixel-based thresholding fails to align shading regions with object semantics, producing fragmented or inconsistent shadow areas. In contrast, our region-wise strategy performs local thresholding within each semantic mask, ensuring that shadow boundaries conform to the geometry of individual objects. This alignment results in cleaner initialization and more accurate structural optimization.

区域级语义二值化。 为获得具有明确结构意义的照明层,我们比较了四种二值化策略:自适应阈值、OTSU、全局阈值,以及我们提出的区域级语义二值化策略。 如 图12 所示,传统的基于像素的阈值方法无法使阴影区域与对象语义对齐,产生碎片化或不一致的阴影区域。 相比之下,我们的区域级策略在每个语义掩码内部执行局部阈值处理,确保阴影边界符合各个对象的几何结构。 这种对齐带来了更清晰的初始化与更准确的结构优化。

We also incorporate region-wise semantic binarization into the initialization of the simplified setting for emoji-style images. As shown in Figure 13, this strategy preserves finer details and produces cleaner shapes.

我们还将区域级语义二值化引入表情符号风格图像的简化配置初始化中。 如 图13 所示,该策略能够保留更精细的细节,并生成更清晰的形状。

用于照明初始化的二值化策略比较。包括全局阈值、OTSU 和自适应阈值在内的传统二值化方法,基于全局或局部强度分布进行像素级阈值处理。
图12:用于照明初始化的二值化策略比较。包括全局阈值、OTSU 和自适应阈值在内的传统二值化方法,基于全局或局部强度分布进行像素级阈值处理。
在相同路径预算下,区域级语义二值化对表情符号矢量化结果的影响。
图13:在相同路径预算下,区域级语义二值化对表情符号矢量化结果的影响。

Structural Optimization. Figure 14 evaluates the effect of removing each component of our optimization objective in structural optimization. Without the reconstruction loss Lrecon, the model fails to recover accurate colors, yielding visually inconsistent results despite relatively stable geometry. On the other hand, removing the structure loss Lstruct leads to irregular and unstable vector boundaries, causing fragmented regions and degraded semantic coherence, although the overall color tone is preserved. Our full objective contains both losses, yielding smooth and well-organized vector geometry together with faithful color that best matches the target image. This joint formulation provides a stable foundation for subsequent refinement.

结构优化。 图14 评估了在结构优化中移除优化目标各组成部分的影响。 若去掉重建损失 Lrecon,模型无法恢复准确颜色,尽管几何结构相对稳定,结果在视觉上仍不一致。 另一方面,若去掉结构损失 Lstruct,则会出现不规则且不稳定的矢量边界,造成区域碎片化与语义连贯性下降,尽管整体色调得以保留。 完整目标同时包含这两种损失,能够生成平滑、组织良好的矢量几何结构,并忠实还原与目标图像最匹配的颜色。 这一联合建模为后续细化提供了稳定的基础。

结构优化中结构损失与重建损失的消融实验。
图14:结构优化中结构损失与重建损失的消融实验。

6. Conclusion ​

We propose COVec, an illumination-aware method for real-world image vectorization that integrates the artistic principle of Clair-Obscur into the vector domain. COVec bridges intrinsic image decomposition with geometric vectorization, explicitly decoupling albedo, shade, and light into distinct layers within a unified vector representation. We propose a region-wise semantic binarization strategy that serves as an effective initialization for illumination modeling. Extensive experiments demonstrate that our method achieves superior visual fidelity, more semantically concise layer organization, and stronger editability compared with existing approaches.

我们提出了 COVec,一种面向真实世界图像的光照感知矢量化方法,将明暗对照的艺术原理融入矢量域。 COVec 将本征图像分解与几何矢量化相结合,在统一的矢量表示中,将反照率、阴影和光照显式解耦为不同图层。 我们提出区域级语义二值化策略,为照明建模提供有效的初始化。 大量实验表明,与现有方法相比,我们的方法具有更高的视觉保真度、语义上更简洁的图层组织,以及更强的可编辑性。