1. Introduction

Edges encode topological and photometric transitions that organize scene understanding and often act as priors for segmentation, contour completion, optical flow, and text/lesion boundary extraction. Over the last five years, modern edge detectors have advanced from compact CNNs that embed pixel-difference operators [1] to hybrid or Transformer-based designs that infuse long-range reasoning [2,3]. While global self-attention improves boundary continuity, its quadratic complexity scales poorly on high-resolution inputs. Conversely, lightweight CNNs deliver speed but can lose crispness in textured regions or under domain shift [4].

Two converging observations motivate EdgeNAT. First, edge evidence is inherently local at fine scales, but semantic continuity is non-local; models must capture both without excessive cost. Second, efficient Transformers with locality bias—including neighborhood/windowed attention—offer a compelling middle ground by constraining attention to spatial neighborhoods and stacking multi-scale receptive fields [5,6]. Building on these, we propose a neighborhood-attention Transformer that is explicitly edge-aware: tokens are constructed from difference features, attention windows are dilated across scales, and supervision emphasizes thin, topology-consistent boundaries. Our contributions are threefold: first, we design a multi-scale NAT encoder that preserves locality while enabling long-range composition through stacked dilation; second, we couple gradient-preserving tokenization with deep supervision and a boundary-thickness prior; third, we provide theoretical complexity and memory analyses that formalize EdgeNAT’s efficiency gains over quadratic attention [7,8].

2. Related work

Recent journal works push precision while balancing efficiency. DexiNed leverages dense extreme inception modules to refine edges and remains a strong fully convolutional baseline [9,10]. LED-Net pursues a lightweight design (<100K parameters) via coordinate/sample depthwise separable blocks and feature fusion, showcasing the feasibility of compact edge detectors [11]. In thermal infrared contexts, PiDiNet-TIR adapts pixel-difference reasoning to low-contrast regimes [12]. Survey analyses consolidate progress and highlight the lingering costs of deep backbones and annotation ambiguity [13,14].

Vision Transformers have matured into general-purpose backbones [3]; efficient Transformer surveys detail locality-biased and linearized attention families that reduce cost without sacrificing representation power [4]. Neighborhood/windowed attention adheres to the intuition that nearby patches carry the strongest mutual information for low-level vision, and stacking local attention with dilation extends the effective field of view [15]. Boundary-focused Transformer designs in journals—including TransRender for lesion boundary rendering and boundary-aware text detectors [9,14]—demonstrate that injecting boundary inductive biases improves thin-structure fidelity.

Positioning. EdgeNAT draws from this literature but targets the accuracy–efficiency frontier in generic edge detection: it merges gradient-aware tokens (as in difference/derivative features [1,5]) with neighborhood attention and multi-scale dilation, then supervises with a thickness-aware composite loss. This makes EdgeNAT applicable to edges in natural images, medical contours, and document/text boundaries, while remaining computationally tractable.

3. Method

3.1. Overview

EdgeNAT comprises three stages: (i) a convolutional stem that computes pixel-difference and low-level features; (ii) a pyramidal NAT encoder with stages at  $$ \frac{1}{4}$$ ,  $$ \frac{1}{8}$$ , and  $$ \frac{1}{16}$$  resolution, where each stage stacks  $$ L_s$$  Neighborhood Attention blocks using window size  $$ k\times k$$  and dilation  $$ d_s$$ ; (iii) a multi-branch decoder with lateral connections and deep supervision on side outputs. The stem converts an image  $$ I\in {\mathbb{R}}^{H\times W\times 3}$$  into tokens that retain gradient cues. Neighborhood attention at location  $$ p$$  attends over a local set  $$ {\mathcal{N}}_k\left(p\right)$$  defined by window  $$ k$$  and dilation  $$ d$$ :  $$ \mathcal{ }\mathcal{N}{k}^{\left(d\right)}\left(p\right)\mathrm{ }=\mathrm{ }\mathrm{ }q:\mathrm{ }\left|p\mathrm{ }-\mathrm{ }q\right|\mathrm{\infty }\mathrm{ }\le \mathrm{ }\backslash \mathrm{t}\mathrm{f}\mathrm{r}\mathrm{a}\mathrm{c}k-12\cdot \mathrm{ }d$$ . Stacking larger dilations across stages yields a multi-scale composition of local-to-regional dependencies, approximating global coherence with linear cost in  $$ N=\mathrm{H}\mathrm{W}$$ .

3.2. Gradient-preserving tokenization

Edge detectors benefit from tokens that encode contrastive structure. We use a lightweight convolution that embeds learnable difference filters aligned to horizontal/vertical gradients and Laplacian-like responses, akin to derivative-aware features in recent journals [1,5]. Let  $$ f_0$$  denote stem features; tokens  $$ t$$  are projections of  $$ [f_0,{\nabla }_{x}I,{\nabla }_{y}I,\mathrm{\Delta }I]$$ , normalized to stabilize attention logits for thin structures.

3.3. Neighborhood attention with dilation

For queries  $$ Q$$ , keys  $$ K$$ , values  $$ V$$ , standard attention computes  $$ \mathrm{ }\text{softmax}\left(Q{K}^{\mathrm{\top }}\mathrm{ }/\mathrm{ }\sqrt{d}\right)V$$  over all tokens. EdgeNAT restricts keys to  $$ {\mathcal{N}}_k^{\left(d\right)}\left(p\right)$$ , giving

$$$ \mathrm{A}\mathrm{t}\mathrm{t}\mathrm{n}\left(p\right)=\sum _{q\in {\mathcal{N}}_k^{\left(d\right)}\left(p\right)} \frac{\mathrm{e}\mathrm{x}\mathrm{p}(⟨Q_p,K_q⟩/\sqrt{d})}{\sum _{r\in {\mathcal{N}}_k^{\left(d\right)}\left(p\right)} \mathrm{e}\mathrm{x}\mathrm{p}(⟨{\mathrm{Q}}_{\mathrm{p}},{\mathrm{K}}_{\mathrm{r}}⟩/\sqrt{\mathrm{d}})}\mathrm{ }{\mathrm{V}}_{\mathrm{q}}.$$$(1)

This yields complexity  $$ \mathcal{O}(N\cdot M)$$  where  $$ M=k^2$$  per head, instead of  $$ \mathcal{O}\left(N^2\right)$$ . By increasing dilation  $$ d$$  across stages, the model captures long-range trends with bounded local computations, resonating with journal findings on locality-biased attention in remote sensing and medical imaging [5,6,8].

3.4. Decoder and deep supervision

We upsample encoder features with lateral concatenation and produce side outputs at each scale. Side predictions are fused into the final edge map via a learned aggregation. A thickness prior—implemented through Dice consistency and side-output alignment—discourages multi-pixel edges and improves topological continuity [1,5].

3.5. Loss function

Let  $$ y\in \{\mathrm{0,1}{\}}^{H\times W}$$  be the ground-truth edge map and  $$ {\hat{y}}_s$$  the side prediction at scale  $$ s$$ . With class-imbalance weight  $$ \alpha $$  and side weights  $$ {\lambda }_s$$ , the composite loss is

$$$ \mathcal{ }\mathcal{L}\mathcal{ }=\mathcal{ }\sum _s {\lambda }_s\left[\alpha \mathrm{ }\mathrm{B}\mathrm{C}\mathrm{E}\right(y,{\hat{y}}_s)+(1-\alpha \left)\right(1-\frac{2⟨y,{\hat{y}}_s⟩+ϵ}{\Vert y{\Vert }_2^2+\Vert {\hat{y}}_s{\Vert }_2^2+\mathrm{ϵ}}\left)\right]\mathrm{ }+\mathrm{ }\mathrm{\gamma }\mathrm{ }\mathrm{I}\mathrm{o}\mathrm{U}(\mathrm{y},{\stackrel{\mathrm{ˆ}}{\mathrm{y}}}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}})\mathrm{ }$$$(2)

where the Dice-like term enforces thin, overlap-consistent boundaries; IoU on the fused output stabilizes late fusion [1,5].

4. Results and discussion

4.1. Theoretical complexity and memory

The principal motivation for EdgeNAT is to control the attention neighborhood. For an  $$ H\times W$$  image with  $$ N=HW$$  tokens:

Global attention: time and memory scale as  $$ \mathcal{O}\left(N^2\right)$$ .

Neighborhood/windowed attention (EdgeNAT):  $$ \mathcal{ }\mathcal{O}\left(N\cdot \mathrm{ }M\right)$$  with  $$ M=k^2$$  independent of  $$ N$$ . For fixed  $$ k$$ , the gap grows linearly with resolution [3,4,6,8].

Stacked dilation: provides an effective receptive field larger than  $$ k$$  without changing  $$ M$$ , encouraging boundary continuity at low marginal cost.

Figure 1 visualizes a typical neighborhood kernel (left) and plots memory growth against sequence length (right) for global versus neighborhood attention. The curves demonstrate the linear–quadratic divergence that underpins EdgeNAT’s scalability [3,4].

The neighborhood kernel emphasizes local affinity that decays with distance, matching the inductive bias of edges as thin, locally coherent structures. The memory plot quantifies why global attention becomes untenable for megapixel inputs, while EdgeNAT scales linearly in  $$ N$$  [3,4].

4.2. Efficiency landscape of neighborhood size

Figure 2 provides a heatmap of the complexity ratio  $$ \rho (N,k)=\frac{N\cdot k^2}{N^2}=k^2/N$$  between neighborhood and global attention across resolutions and window sizes. For typical edge inputs (e.g.,  $$ H=W=512$$ ),  $$ k=7$$  yields  $$ \rho \approx 49/\mathrm{262,144}\approx 1.9\times {10}^{-4}$$ , indicating orders-of-magnitude savings with negligible locality loss once stacked across dilations [4,6,8].

The heatmap shows that even moderately sized windows keep  $$ \rho $$  extremely small at realistic resolutions. Empirical reports on neighborhood attention in journals corroborate that locality-biased attention recovers global structure when stacked with multi-scale dilation [6,8].

4.3. Architectural choices and literature alignment

First, gradient-aware tokens stabilize attention over thin structures and reduce over-smoothing, consistent with derivative-infused backbones used for boundary detection and lesion rendering [1,5]. Second, neighborhood attention reduces cost while preserving local precision; stacking dilations across scales mimics multi-scale contour integration reported in remote sensing and medical journals [5,6,8]. Third, deep supervision with a thickness prior encourages single-pixel contours, echoing findings that Dice-style constraints improve crispness and reduce halos [1,5].

Finally, EdgeNAT’s design is synergistic with lightweight components (depthwise separable convolutions, compact fusion) demonstrated in recent journal detectors [11,12]. Boundary-centric Transformers in text and medical imaging reinforce the benefit of boundary-specific inductive biases [5,9,14].

With fixed k, neighborhood attention is linear in N. Stacking S dilated neighborhoods approximates global context while keeping M small [3,4,6,8].

5. Conclusion

We introduced EdgeNAT, a Transformer-based edge detector that reconciles crisp localization with computational efficiency via neighborhood attention, gradient-preserving tokenization, and thickness-aware deep supervision. Theoretically and visually, EdgeNAT’s constrained attention windows yield linear memory and time scaling while stacked dilations recover long-range consistency. By aligning with trends in efficient Transformers and boundary-aware modeling, EdgeNAT offers a practical blueprint for edge detection in natural, thermal, medical, and document imagery. Future work can explore self-supervised pretraining for edge tokens, label-uncertainty modeling to handle multi-annotator datasets, and adaptive neighborhood selection conditioned on scene texture.