{"title":"","byline":null,"dir":null,"lang":null,"content":"

\"Version\"\n\"Python\"\n\"License\"\n\"OS\"

\n

Puppet Warp

\n

The goal of the package puppet-warp is to provide a plug-and-play solution for image\ntransformation similar to Adobe Photoshop’s Puppet Warp tool. Since the Photoshop\nsolution is proprietary (and scripting can be painful, especially on unsupported platforms),\nthis project implements Puppet Warp in Python so it can be used programmatically in\nautomation pipelines where advanced deformation is required.

\n

Features

\n
    \n
  • As-Rigid-as-Possible (ARAP) shape manipulation of a triangular mesh
    • \n
  • Image transfer from a triangular mesh at rest to a mesh defined by ARAP deformation
    • \n
\n
\n

Note: Please report issues and feel free to open pull requests.

\n
\n

Requirements

\n
numpy>=1.21.5\nopencv-contrib-python>=4.5.4.60,<=4.12.0.88\nopencv-python>=4.5.4.60,<=4.12.0.88\nscikit-image>=0.19.2,<=0.26.0\nscikit-learn>=1.0.2,<=1.8.0\n
\n

Optional:

\n\n

Installation

\n\n

For the latest version from git:

\n
pip install git+https://github.com/mikecokina/puppet-warp.git@dev
\n

Install with Jonathan Richard Shewchuk’s Triangle bindings:

\n
pip install puppet-warp[jrs]
\n

Usage

\n

Demo

\n

The package comes with a live interactive demo:

\n
from pwarp import Demo\n\nDemo().run()
\n

To manipulate the image:

\n
    \n
  • Select control points by clicking on vertices in the mesh.
    • \n
  • Drag a selected control point to deform the mesh.
    • \n
\n

Demo also supports saving the transformed mesh:

\n
    \n
  • Space: save current mesh (Wavefront OBJ)
    • \n
  • Esc: quit
    • \n
\n

By default, outputs are stored in ~/pwarp.

\n

Custom demo

\n
import cv2\n\nfrom pwarp import Demo, triangular_mesh\nfrom pwarp._io import save_wavefront\n\n# Define WIDTH and HEIGHT of your image and DELTA step to create a triangular mesh.\nwidth = 800\nheight = 492\ndelta = 100\nmethod = \"scipy\"  # or \"jrs\"\n\n# Define paths to your image and the OBJ file.\nwavefront_path = \"image.obj\"\nimage_path = \"image.jpg\"\n\nimage = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n\n# Generate triangular mesh over the image.\nr, f = triangular_mesh(width=width, height=height, delta=delta, method=method)\n\n# Save wavefront object.\nsave_wavefront(wavefront_path, no_vertices=len(r), no_faces=len(f), vertices=r, faces=f)\n\nDemo(\n    image=image_path,\n    obj_path=wavefront_path,\n    screen_height=height,\n    screen_width=width,\n    scale=1,\n    dx=0,\n    dy=0,\n    verbose=True,\n).run()
\n

Graph warp

\n

Graph warp requires vertices and faces (triangulation), control points, and new positions of\ncontrol points. Based on that information, graph warp computes new positions of the supplied\nvertices.

\n

Example:

\n
import numpy as np\n\nfrom pwarp import get_default_puppet, graph_warp\nfrom pwarp.core.precompute import arap_precompute\n\n# Control points represent indices of points in original vertex array.\ncontrol_pts = np.array([22, 50, 94, 106], dtype=int)\n\n# Shift represents new positions of control points respectively to `control_pts` list.\nshift = np.array(\n    [\n        [0.555, -0.905],\n        [-0.965, -0.875],\n        [-0.950, 0.460],\n        [0.705, 0.285],\n    ],\n    dtype=float,\n)\n\npuppet = get_default_puppet()\n\n# Precompute once per mesh (recommended).\npre = arap_precompute(vertices=puppet.r, faces=puppet.f)\n\nnew_vertices = graph_warp(\n    vertices=puppet.r,\n    faces=puppet.f,\n    control_indices=control_pts,\n    shifted_locations=shift,\n    precomputed=pre,\n)
\n

\"mesh\"

\n

Graph defined warp

\n

Graph defined warp transforms image regions covered by source vertices to given destination vertices.\nIt requires:

\n
    \n
  • input image
    • \n
  • source vertices + faces
    • \n
  • destination vertices + faces
    • \n
\n

Faces (triangles) must correspond pairwise between source and destination.

\n

Example:

\n
import cv2\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nfrom pwarp import get_default_puppet, graph_defined_warp, graph_warp\nfrom pwarp.core.precompute import arap_precompute\n\ncontrol_pts = np.array([22, 50, 94, 106], dtype=int)\nshift = np.array(\n    [\n        [0.555, -0.905],\n        [-0.965, -0.875],\n        [-0.950, 0.460],\n        [0.705, 0.285],\n    ],\n    dtype=float,\n)\n\npuppet = get_default_puppet()\npre = arap_precompute(vertices=puppet.r, faces=puppet.f)\n\nnew_r = graph_warp(\n    vertices=puppet.r,\n    faces=puppet.f,\n    control_indices=control_pts,\n    shifted_locations=shift,\n    precomputed=pre,\n)\n\nimage = cv2.cvtColor(cv2.imread(\"../data/puppet.png\"), cv2.COLOR_BGR2RGB)\nwidth, height = 1280, 800\ndx, dy = int(width // 2), int(height // 2)\nscale_x, scale_y = 200, -200\n\nr = puppet.r.copy()\nr[:, 0] = r[:, 0] * scale_x + dx\nr[:, 1] = r[:, 1] * scale_y + dy\n\nnew_r = new_r.copy()\nnew_r[:, 0] = new_r[:, 0] * scale_x + dx\nnew_r[:, 1] = new_r[:, 1] * scale_y + dy\n\nimage_t = graph_defined_warp(\n    image,\n    vertices_src=r,\n    faces_src=puppet.f,\n    vertices_dst=new_r,\n    faces_dst=puppet.f,\n)\n\nfig, axs = plt.subplots(1, 2, frameon=False)\nplt.tight_layout(pad=0)\n\naxs[0].imshow(image)\naxs[1].imshow(image_t)\naxs[0].triplot(r.T[0], r.T[1], puppet.f, lw=0.5)\naxs[1].triplot(new_r.T[0], new_r.T[1], puppet.f, lw=0.5)\n\nfor ax in axs:\n    ax.set_xlim([380, 900])\n    ax.set_ylim([150, 750])\n    ax.invert_yaxis()\n    ax.axis(\"off\")\n\nplt.show()
\n

\"mesh\"

\n

Triangular mesh

\n

The algorithm generates a triangular mesh within a rectangle defined by its width and height.\nMesh density is adjustable via the delta parameter.

\n

Example:

\n
from pwarp import triangular_mesh\n\nr, f = triangular_mesh(width=1280, height=800, delta=100)
\n

\"mesh\"

\n

Example on full screen triangular mesh warp:

\n

\"mesh\"

\n

References

\n
[1] https://www-ui.is.s.u-tokyo.ac.jp/~takeo/papers/takeo_jgt09_arapFlattening.pdf\n[2] https://github.com/deliagander/ARAPShapeManipulation.git\n[3] https://learnopencv.com/warp-one-triangle-to-another-using-opencv-c-python/\n[4] https://rufat.be/triangle/\n[5] http://www.cs.cmu.edu/~quake/triangle.html\n
\n

Cite

\n
@article{journals/jgtools/IgarashiI09,\n    author = {Igarashi, Takeo and Igarashi, Yuki},\n    ee = {http://dx.doi.org/10.1080/2151237X.2009.10129273},\n    journal = {J. Graphics, GPU, & Game Tools},\n    number = 1,\n    pages = {17-30},\n    title = {Implementing As-Rigid-As-Possible Shape Manipulation and Surface Flattening.},\n    url = {http://dblp.uni-trier.de/db/journals/jgtools/jgtools14.html#IgarashiI09},\n    volume = 14,\n    year = 2009\n}
\n

or

\n
@article{10.1145/1073204.1073323,\n    author = {Igarashi, Takeo and Moscovich, Tomer and Hughes, John F.},\n    title = {As-Rigid-as-Possible Shape Manipulation},\n    year = {2005},\n    publisher = {Association for Computing Machinery},\n    address = {New York, NY, USA},\n    volume = {24},\n    number = {3},\n    doi = {10.1145/1073204.1073323},\n    journal = {ACM Trans. Graph.},\n    month = {jul},\n    pages = {1134–1141},\n    numpages = {8}\n}
\n
","textContent":"\n\n\n\nPuppet Warp\nThe goal of the package puppet-warp is to provide a plug-and-play solution for image\ntransformation similar to Adobe Photoshop’s Puppet Warp tool. Since the Photoshop\nsolution is proprietary (and scripting can be painful, especially on unsupported platforms),\nthis project implements Puppet Warp in Python so it can be used programmatically in\nautomation pipelines where advanced deformation is required.\nFeatures\n\nAs-Rigid-as-Possible (ARAP) shape manipulation of a triangular mesh\nImage transfer from a triangular mesh at rest to a mesh defined by ARAP deformation\n\n\nNote: Please report issues and feel free to open pull requests.\n\nRequirements\nnumpy>=1.21.5\nopencv-contrib-python>=4.5.4.60,<=4.12.0.88\nopencv-python>=4.5.4.60,<=4.12.0.88\nscikit-image>=0.19.2,<=0.26.0\nscikit-learn>=1.0.2,<=1.8.0\n\nOptional:\n\nInstallation\n\nFor the latest version from git:\npip install git+https://github.com/mikecokina/puppet-warp.git@dev\nInstall with Jonathan Richard Shewchuk’s Triangle bindings:\npip install puppet-warp[jrs]\nUsage\nDemo\nThe package comes with a live interactive demo:\nfrom pwarp import Demo\n\nDemo().run()\nTo manipulate the image:\n\nSelect control points by clicking on vertices in the mesh.\nDrag a selected control point to deform the mesh.\n\nDemo also supports saving the transformed mesh:\n\nSpace: save current mesh (Wavefront OBJ)\nEsc: quit\n\nBy default, outputs are stored in ~/pwarp.\nCustom demo\nimport cv2\n\nfrom pwarp import Demo, triangular_mesh\nfrom pwarp._io import save_wavefront\n\n# Define WIDTH and HEIGHT of your image and DELTA step to create a triangular mesh.\nwidth = 800\nheight = 492\ndelta = 100\nmethod = \"scipy\" # or \"jrs\"\n\n# Define paths to your image and the OBJ file.\nwavefront_path = \"image.obj\"\nimage_path = \"image.jpg\"\n\nimage = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n\n# Generate triangular mesh over the image.\nr, f = triangular_mesh(width=width, height=height, delta=delta, method=method)\n\n# Save wavefront object.\nsave_wavefront(wavefront_path, no_vertices=len(r), no_faces=len(f), vertices=r, faces=f)\n\nDemo(\n image=image_path,\n obj_path=wavefront_path,\n screen_height=height,\n screen_width=width,\n scale=1,\n dx=0,\n dy=0,\n verbose=True,\n).run()\nGraph warp\nGraph warp requires vertices and faces (triangulation), control points, and new positions of\ncontrol points. Based on that information, graph warp computes new positions of the supplied\nvertices.\nExample:\nimport numpy as np\n\nfrom pwarp import get_default_puppet, graph_warp\nfrom pwarp.core.precompute import arap_precompute\n\n# Control points represent indices of points in original vertex array.\ncontrol_pts = np.array([22, 50, 94, 106], dtype=int)\n\n# Shift represents new positions of control points respectively to `control_pts` list.\nshift = np.array(\n [\n [0.555, -0.905],\n [-0.965, -0.875],\n [-0.950, 0.460],\n [0.705, 0.285],\n ],\n dtype=float,\n)\n\npuppet = get_default_puppet()\n\n# Precompute once per mesh (recommended).\npre = arap_precompute(vertices=puppet.r, faces=puppet.f)\n\nnew_vertices = graph_warp(\n vertices=puppet.r,\n faces=puppet.f,\n control_indices=control_pts,\n shifted_locations=shift,\n precomputed=pre,\n)\n\nGraph defined warp\nGraph defined warp transforms image regions covered by source vertices to given destination vertices.\nIt requires:\n\ninput image\nsource vertices + faces\ndestination vertices + faces\n\nFaces (triangles) must correspond pairwise between source and destination.\nExample:\nimport cv2\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nfrom pwarp import get_default_puppet, graph_defined_warp, graph_warp\nfrom pwarp.core.precompute import arap_precompute\n\ncontrol_pts = np.array([22, 50, 94, 106], dtype=int)\nshift = np.array(\n [\n [0.555, -0.905],\n [-0.965, -0.875],\n [-0.950, 0.460],\n [0.705, 0.285],\n ],\n dtype=float,\n)\n\npuppet = get_default_puppet()\npre = arap_precompute(vertices=puppet.r, faces=puppet.f)\n\nnew_r = graph_warp(\n vertices=puppet.r,\n faces=puppet.f,\n control_indices=control_pts,\n shifted_locations=shift,\n precomputed=pre,\n)\n\nimage = cv2.cvtColor(cv2.imread(\"../data/puppet.png\"), cv2.COLOR_BGR2RGB)\nwidth, height = 1280, 800\ndx, dy = int(width // 2), int(height // 2)\nscale_x, scale_y = 200, -200\n\nr = puppet.r.copy()\nr[:, 0] = r[:, 0] * scale_x + dx\nr[:, 1] = r[:, 1] * scale_y + dy\n\nnew_r = new_r.copy()\nnew_r[:, 0] = new_r[:, 0] * scale_x + dx\nnew_r[:, 1] = new_r[:, 1] * scale_y + dy\n\nimage_t = graph_defined_warp(\n image,\n vertices_src=r,\n faces_src=puppet.f,\n vertices_dst=new_r,\n faces_dst=puppet.f,\n)\n\nfig, axs = plt.subplots(1, 2, frameon=False)\nplt.tight_layout(pad=0)\n\naxs[0].imshow(image)\naxs[1].imshow(image_t)\naxs[0].triplot(r.T[0], r.T[1], puppet.f, lw=0.5)\naxs[1].triplot(new_r.T[0], new_r.T[1], puppet.f, lw=0.5)\n\nfor ax in axs:\n ax.set_xlim([380, 900])\n ax.set_ylim([150, 750])\n ax.invert_yaxis()\n ax.axis(\"off\")\n\nplt.show()\n\nTriangular mesh\nThe algorithm generates a triangular mesh within a rectangle defined by its width and height.\nMesh density is adjustable via the delta parameter.\nExample:\nfrom pwarp import triangular_mesh\n\nr, f = triangular_mesh(width=1280, height=800, delta=100)\n\nExample on full screen triangular mesh warp:\n\nReferences\n[1] https://www-ui.is.s.u-tokyo.ac.jp/~takeo/papers/takeo_jgt09_arapFlattening.pdf\n[2] https://github.com/deliagander/ARAPShapeManipulation.git\n[3] https://learnopencv.com/warp-one-triangle-to-another-using-opencv-c-python/\n[4] https://rufat.be/triangle/\n[5] http://www.cs.cmu.edu/~quake/triangle.html\n\nCite\n@article{journals/jgtools/IgarashiI09,\n author = {Igarashi, Takeo and Igarashi, Yuki},\n ee = {http://dx.doi.org/10.1080/2151237X.2009.10129273},\n journal = {J. Graphics, GPU, & Game Tools},\n number = 1,\n pages = {17-30},\n title = {Implementing As-Rigid-As-Possible Shape Manipulation and Surface Flattening.},\n url = {http://dblp.uni-trier.de/db/journals/jgtools/jgtools14.html#IgarashiI09},\n volume = 14,\n year = 2009\n}\nor\n@article{10.1145/1073204.1073323,\n author = {Igarashi, Takeo and Moscovich, Tomer and Hughes, John F.},\n title = {As-Rigid-as-Possible Shape Manipulation},\n year = {2005},\n publisher = {Association for Computing Machinery},\n address = {New York, NY, USA},\n volume = {24},\n number = {3},\n doi = {10.1145/1073204.1073323},\n journal = {ACM Trans. Graph.},\n month = {jul},\n pages = {1134–1141},\n numpages = {8}\n}\n","length":6501,"excerpt":"","siteName":null}