Бесплатный парсер товаров онлайн магазина Alexander McQueen

Posted on: Декабрь 30, 2017 /

Categories: Бесплатные парсеры, Парсер магазина

Alexander McQueen — был известным британским модным дизайнером и основал свой лейбл и модный дом. Этот парсер товаров онлайн магазина предназначен для сбора данных о товарах с флагманского интернет магазина модного дома alexandermcqueen.com.

Примерное количество товаров: 1000
Примерное количество запросов: 2300
Рекомендуемый план подписки: Free

ВНИМАНИЕ! Количество запросов может превышать количество товаров, потому что данные о вариациях, изображениях и др. могут парсится используя запросы к дополнительным ресурсам. Также часть данных о товаре может доставляться с помощью XHR запросов, что также увеличивает общее количество необходимых запросов.

Для его использования вы должны иметь учетную запись в нашем сервисе Diggernaut.

Пройдите по этой ссылке для регистрации в сервисе Diggernaut
После регистрации и подтверждения email адреса войдите в свою учетную запись
Создайте проект с любый именем и описанием, если вы не знаете как, обратитесь к нашей документации
Войдите во вновь созданный проект и создайте в нем диггер с любым именем, если вы не знаете как, обратитесь к нашей документации
Скопируйте в буфер обмена приведенный ниже сценарий диггера и вставьте его в созданный вами диггер, если вы не знаете как, обратитесь к нашей документации
Переключите режим работы диггера с Debug на Active, если вы не знаете как, обратитесь к нашей документации
Запустите ваш диггер и дождитесь окончания его работы, если вы не знаете как, обратитесь к нашей документации
Скачайте собранный набор данных в нужном вам формате, если вы не знаете как, обратитесь к нашей документации

В дальнейшем вы можете установить расписание для запуска вашего парсера и забирать информацию регулярно.

Сценарий парсера:

---
config:
    debug: 2
    agent: Firefox
do:
- walk:
    to: http://www.alexandermcqueen.com/us/
    do:
    - find:
        path: ul.level-1>li
        do:
        - variable_clear: cat1
        - parse:
            attr: id
        - normalize:
            routine: replace_matched
            args:
                shop_womenswear: Womens
                shop_menswear: Mens
                .+: ''
        - variable_set: cat1
        - find:
            path: ul.level-2>li
            do:
            - variable_clear: cat2
            - find:
                path: a
                slice: 0
                do:
                - parse
                - space_dedupe
                - trim
                - variable_set: cat2
            - find:
                path: ul.level-3>li
                do:
                - variable_clear: cat3
                - find:
                    path: a
                    slice: 0
                    do:
                    - parse
                    - space_dedupe
                    - trim
                    - variable_set: cat3
                    - parse:
                        attr: href
                    - space_dedupe
                    - trim
                    - if:
                        match: \w+
                        do:
                        - normalize:
                            routine: url
                        - walk:
                            to: value
                            do:
                            - find:
                                path: script:contains('yTos.navigation =')
                                do:
                                - parse:
                                    filter: yTos\.navigation\s+\=\s+(.+)\;
                                - normalize:
                                    routine: json2xml
                                - to_block
                                - find:
                                    path: pathandqueryparsed:has(paramname:matches(^sitecode$))
                                    do:
                                    - find:
                                        path: paramvalue
                                        do:
                                        - parse
                                        - variable_set: sitecode
                                - find:
                                    path: pathandqueryparsed:has(paramname:matches(^dept$))
                                    do:
                                    - find:
                                        path: paramvalue
                                        do:
                                        - parse
                                        - variable_set: department
                                - find:
                                    path: pathandqueryparsed:has(paramname:matches(^season$))
                                    do:
                                    - find:
                                        path: paramvalue
                                        do:
                                        - parse
                                        - normalize:
                                            routine: replace_substring
                                            args:
                                                \,: "%2C"
                                        - variable_set: season
                                - find:
                                    path: pathandqueryparsed:has(paramname:matches(^gender$))
                                    do:
                                    - find:
                                        path: paramvalue
                                        do:
                                        - parse
                                        - variable_set: gender
                                - find:
                                    path: pathandqueryparsed:has(paramname:matches(^yurirulename$))
                                    do:
                                    - find:
                                        path: paramvalue
                                        do:
                                        - parse
                                        - variable_set: yurirulename
                                - walk:
                                    to: http://www.alexandermcqueen.com/Search/RenderProducts?ytosQuery=true&department=<%department%>&gender=<%gender%>&season=<%season%>&yurirulename=<%yurirulename%>&page=1&productsPerPage=1000&suggestion=false&totalPages=1&partialLoadedItems=1000&siteCode=<%sitecode%>
                                    do:
                                    - find:
                                        path: article>a
                                        do:
                                        - parse:
                                            attr: href
                                            filter: ^([^\#]+)
                                        - walk:
                                            to: value
                                            do:
                                            - sleep: 2
                                            - find:
                                                path: article.item
                                                do:
                                                - variable_clear: pid
                                                - variable_clear: cid
                                                - object_new: product
                                                - eval:
                                                    routine: js
                                                    body: '(function (){var d = new Date(); return d.toISOString()})();'
                                                - object_field_set:
                                                    object: product
                                                    field: date
                                                - static_get: url
                                                - object_field_set:
                                                    object: product
                                                    field: url
                                                - register_set: Alexander McQueen
                                                - object_field_set:
                                                    object: product
                                                    field: brand
                                                - find:
                                                    path: h2.modelName
                                                    do:
                                                    - parse
                                                    - space_dedupe
                                                    - trim
                                                    - object_field_set:
                                                        object: product
                                                        field: name
                                                - find:
                                                    in: doc
                                                    path: meta[name="description"]
                                                    do:
                                                    - parse:
                                                        attr: content
                                                    - space_dedupe
                                                    - trim
                                                    - variable_set: desc
                                                - find:
                                                    path: div.descriptionsContainer>div.EditorialDescription
                                                    do:
                                                    - parse
                                                    - space_dedupe
                                                    - trim
                                                    - variable_set: desc
                                                - variable_get: desc
                                                - object_field_set:
                                                    object: product
                                                    field: description
                                                - find:
                                                    path: div.itemPriceContainer span.price
                                                    slice: 0
                                                    do:
                                                    - find:
                                                        path: span.currency
                                                        do:
                                                        - parse
                                                        - normalize:
                                                            routine: replace_matched
                                                            args:
                                                                \$: USD
                                                        - object_field_set:
                                                            object: product
                                                            field: currency
                                                    - find:
                                                        path: span.value
                                                        do:
                                                        - parse
                                                        - normalize:
                                                            routine: replace_substring
                                                            args:
                                                            - \,: ''
                                                            - \s+: ''
                                                        - object_field_set:
                                                            object: product
                                                            type: float
                                                            field: price
                                                - find:
                                                    path: div.modelFabricColor>span.value
                                                    do:
                                                    - parse
                                                    - space_dedupe
                                                    - trim
                                                    - if:
                                                        match: \w+
                                                        do:
                                                        - variable_set: pid
                                                        - object_field_set:
                                                            object: product
                                                            field: sku
                                                - variable_get: cat1
                                                - if:
                                                    match: \w{2,}
                                                    do:
                                                    - object_field_set:
                                                        object: product
                                                        joinby: "|"
                                                        field: category
                                                - variable_get: cat2
                                                - if:
                                                    match: \w{2,}
                                                    do:
                                                    - object_field_set:
                                                        object: product
                                                        joinby: "|"
                                                        field: category
                                                - variable_get: cat3
                                                - if:
                                                    match: \w{2,}
                                                    do:
                                                    - object_field_set:
                                                        object: product
                                                        joinby: "|"
                                                        field: category
                                                - find:
                                                    path: ul.alternativeImages>li>img
                                                    do:
                                                    - parse:
                                                        attr: srcset
                                                    - to_block
                                                    - split:
                                                        context: text
                                                        delimiter: \,\s*
                                                    - find:
                                                        path: div.splitted
                                                        slice: 0
                                                        do:
                                                        - parse:
                                                            filter: ^([^\s]+)
                                                        - object_field_set:
                                                            object: product
                                                            joinby: "|"
                                                            field: images
                                                - find:
                                                    path: div.selectColor
                                                    slice: 0
                                                    do:
                                                    - variable_clear: cod10
                                                    - find:
                                                        in: doc
                                                        path: script:contains("yTos.navigation.itemData =")
                                                        do:
                                                        - parse:
                                                            filter: yTos\.navigation\.itemData\s+\=\s+(.+)\;
                                                        - normalize:
                                                            routine: json2xml
                                                        - to_block
                                                        - find:
                                                            path: cod10
                                                            do:
                                                            - parse
                                                            - space_dedupe
                                                            - trim
                                                            - variable_set: cod10
                                                            - walk:
                                                                to: http://www.alexandermcqueen.com/yTos/api/Plugins/ItemPluginApi/GetCombinationsAsync/?siteCode=<%sitecode%>&code10=<%cod10%>
                                                                do:
                                                                - find:
                                                                    path: body_safe>colors
                                                                    do:
                                                                    - find:
                                                                        path: description
                                                                        do:
                                                                        - parse
                                                                        - space_dedupe
                                                                        - trim
                                                                        - if:
                                                                            match: \w+
                                                                            do:
                                                                            - object_field_set:
                                                                                object: product
                                                                                joinby: "|"
                                                                                field: variations
                                                - object_save:
                                                    name: product

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

---

config:

debug: 2

agent: Firefox

do:

- walk:

to: http://www.alexandermcqueen.com/us/

do:

- find:

path: ul.level-1>li

do:

- variable_clear: cat1

- parse:

attr: id

- normalize:

routine: replace_matched

args:

shop_womenswear: Womens

shop_menswear: Mens

.+: ''

- variable_set: cat1

- find:

path: ul.level-2>li

do:

- variable_clear: cat2

- find:

path: a

slice: 0

do:

- parse

- space_dedupe

- trim

- variable_set: cat2

- find:

path: ul.level-3>li

do:

- variable_clear: cat3

- find:

path: a

slice: 0

do:

- parse

- space_dedupe

- trim

- variable_set: cat3

- parse:

attr: href

- space_dedupe

- trim

- if:

match: \w+

do:

- normalize:

routine: url

- walk:

to: value

do:

- find:

path: script:contains('yTos.navigation =')

do:

- parse:

filter: yTos\.navigation\s+\=\s+(.+)\;

- normalize:

routine: json2xml

- to_block

- find:

path: pathandqueryparsed:has(paramname:matches(^sitecode$))

do:

- find:

path: paramvalue

do:

- parse

- variable_set: sitecode

- find:

path: pathandqueryparsed:has(paramname:matches(^dept$))

do:

- find:

path: paramvalue

do:

- parse

- variable_set: department

- find:

path: pathandqueryparsed:has(paramname:matches(^season$))

do:

- find:

path: paramvalue

do:

- parse

- normalize:

routine: replace_substring

args:

\,: "%2C"

- variable_set: season

- find:

path: pathandqueryparsed:has(paramname:matches(^gender$))

do:

- find:

path: paramvalue

do:

- parse

- variable_set: gender

- find:

path: pathandqueryparsed:has(paramname:matches(^yurirulename$))

do:

- find:

path: paramvalue

do:

- parse

- variable_set: yurirulename

- walk:

to: http://www.alexandermcqueen.com/Search/RenderProducts?ytosQuery=true&department=<%department%>&gender=<%gender%>&season=<%season%>&yurirulename=<%yurirulename%>&page=1&productsPerPage=1000&suggestion=false&totalPages=1&partialLoadedItems=1000&siteCode=<%sitecode%>

do:

- find:

path: article>a

do:

- parse:

attr: href

filter: ^([^\#]+)

- walk:

to: value

do:

- sleep: 2

- find:

path: article.item

do:

- variable_clear: pid

- variable_clear: cid

- object_new: product

- eval:

routine: js

body: '(function (){var d = new Date(); return d.toISOString()})();'

- object_field_set:

object: product

field: date

- static_get: url

- object_field_set:

object: product

field: url

- register_set: Alexander McQueen

- object_field_set:

object: product

field: brand

- find:

path: h2.modelName

do:

- parse

- space_dedupe

- trim

- object_field_set:

object: product

field: name

- find:

in: doc

path: meta[name="description"]

do:

- parse:

attr: content

- space_dedupe

- trim

- variable_set: desc

- find:

path: div.descriptionsContainer>div.EditorialDescription

do:

- parse

- space_dedupe

- trim

- variable_set: desc

- variable_get: desc

- object_field_set:

object: product

field: description

- find:

path: div.itemPriceContainer span.price

slice: 0

do:

- find:

path: span.currency

do:

- parse

- normalize:

routine: replace_matched

args:

\$: USD

- object_field_set:

object: product

field: currency

- find:

path: span.value

do:

- parse

- normalize:

routine: replace_substring

args:

- \,: ''

- \s+: ''

- object_field_set:

object: product

type: float

field: price

- find:

path: div.modelFabricColor>span.value

do:

- parse

- space_dedupe

- trim

- if:

match: \w+

do:

- variable_set: pid

- object_field_set:

object: product

field: sku

- variable_get: cat1

- if:

match: \w{2,}

do:

- object_field_set:

object: product

joinby: "|"

field: category

- variable_get: cat2

- if:

match: \w{2,}

do:

- object_field_set:

object: product

joinby: "|"

field: category

- variable_get: cat3

- if:

match: \w{2,}

do:

- object_field_set:

object: product

joinby: "|"

field: category

- find:

path: ul.alternativeImages>li>img

do:

- parse:

attr: srcset

- to_block

- split:

context: text

delimiter: \,\s*

- find:

path: div.splitted

slice: 0

do:

- parse:

filter: ^([^\s]+)

- object_field_set:

object: product

joinby: "|"

field: images

- find:

path: div.selectColor

slice: 0

do:

- variable_clear: cod10

- find:

in: doc

path: script:contains("yTos.navigation.itemData =")

do:

- parse:

filter: yTos\.navigation\.itemData\s+\=\s+(.+)\;

- normalize:

routine: json2xml

- to_block

- find:

path: cod10

do:

- parse

- space_dedupe

- trim

- variable_set: cod10

- walk:

to: http://www.alexandermcqueen.com/yTos/api/Plugins/ItemPluginApi/GetCombinationsAsync/?siteCode=<%sitecode%>&code10=<%cod10%>

do:

- find:

path: body_safe>colors

do:

- find:

path: description

do:

- parse

- space_dedupe

- trim

- if:

match: \w+

do:

- object_field_set:

object: product

joinby: "|"

field: variations

- object_save:

name: product

Ниже приведен пример датасета с несколькими товарами в формате JSON (для наглядности). Датасет может быть скачан и как CSV, XLSX, XML, и любой другой текстовый формат используя темплейтный подход.

[{
    "product": {
        "brand": "Alexander McQueen",
        "category": "Shop by|Sale",
        "currency": "USD",
        "date": "2017-12-05T17:25:41.800Z",
        "description": "Sleeveless black leather midi-length dress, in raw cut panels whip-stitched together. Featuring hand-applied, silver-plated metal eyelets, which are hand-laced with multicolored leather laces, left long as decorative fringe.",
        "images": "https://cdn.yoox.biz/items/34/34767065be_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_a.jpg",
        "name": "Whip-Stitched Leather Dress",
        "price": 5699,
        "sku": "493349Q5HLU1666",
        "url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767065be.html",
        "variations": "BLACK"
    }
}
,{
    "product": {
        "brand": "Alexander McQueen",
        "category": "Shop by|Sale",
        "currency": "USD",
        "date": "2017-12-05T17:25:45.343Z",
        "description": "Long-sleeved, crew neck BouclГ© knit dress with multicolored leather laces, laced through hand-applied silver-plated metal eyelets.",
        "images": "https://cdn.yoox.biz/items/34/34767074av_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_a.jpg",
        "name": "BouclГ© Knit Long Dress with Leather Lacing",
        "price": 3179,
        "sku": "493382Q1WHI1666",
        "url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767074av.html",
        "variations": "BLACK"
    }
}
,{
    "product": {
        "brand": "Alexander McQueen",
        "category": "Shop by|Sale",
        "currency": "USD",
        "date": "2017-12-05T17:25:48.261Z",
        "description": "Long ivory plissГ© knit dress with extra fine merino wool piping in red, finished with a decorative lurex cross stitch that is left long as frayed fringe on hem and shoulders. Roll neck and invisible zipper on center back.",
        "images": "https://cdn.yoox.biz/items/34/34767073al_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_a.jpg",
        "name": "Long Knit Dress With Roll Neck",
        "price": 2444,
        "sku": "493374Q1WHC9082",
        "url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767073al.html",
        "variations": "IVORY/RED"
    }
}
,{
    "product": {
        "brand": "Alexander McQueen",
        "category": "Shop by|Sale",
        "currency": "USD",
        "date": "2017-12-05T17:25:51.193Z",
        "description": "Long black plissГ© knit dress with extra fine merino wool piping in red, finished with a decorative lurex cross stitch that is left long as frayed fringe. Featuring a roll neck and long voluminous balloon sleeves with an invisible zipper on center back.",
        "images": "https://cdn.yoox.biz/items/34/34767069qs_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_a.jpg",
        "name": "Long-Sleeved Knit Dress With Roll Neck",
        "price": 2669,
        "sku": "493373Q1WHB1056",
        "url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767069qs.html",
        "variations": "BLACK/RED"
    }
}]

[{

"product": {

"brand": "Alexander McQueen",

"category": "Shop by|Sale",

"currency": "USD",

"date": "2017-12-05T17:25:41.800Z",

"description": "Sleeveless black leather midi-length dress, in raw cut panels whip-stitched together. Featuring hand-applied, silver-plated metal eyelets, which are hand-laced with multicolored leather laces, left long as decorative fringe.",

"images": "https://cdn.yoox.biz/items/34/34767065be_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767065be_18_g_a.jpg",

"name": "Whip-Stitched Leather Dress",

"price": 5699,

"sku": "493349Q5HLU1666",

"url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767065be.html",

"variations": "BLACK"

}

"product": {

"brand": "Alexander McQueen",

"category": "Shop by|Sale",

"currency": "USD",

"date": "2017-12-05T17:25:45.343Z",

"description": "Long-sleeved, crew neck BouclГ© knit dress with multicolored leather laces, laced through hand-applied silver-plated metal eyelets.",

"images": "https://cdn.yoox.biz/items/34/34767074av_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767074av_18_g_a.jpg",

"price": 3179,

"sku": "493382Q1WHI1666",

"url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767074av.html",

"variations": "BLACK"

}

"product": {

"brand": "Alexander McQueen",

"category": "Shop by|Sale",

"currency": "USD",

"date": "2017-12-05T17:25:48.261Z",

"description": "Long ivory plissГ© knit dress with extra fine merino wool piping in red, finished with a decorative lurex cross stitch that is left long as frayed fringe on hem and shoulders. Roll neck and invisible zipper on center back.",

"images": "https://cdn.yoox.biz/items/34/34767073al_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767073al_18_g_a.jpg",

"name": "Long Knit Dress With Roll Neck",

"price": 2444,

"sku": "493374Q1WHC9082",

"url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767073al.html",

"variations": "IVORY/RED"

}

"product": {

"brand": "Alexander McQueen",

"category": "Shop by|Sale",

"currency": "USD",

"date": "2017-12-05T17:25:51.193Z",

"description": "Long black plissГ© knit dress with extra fine merino wool piping in red, finished with a decorative lurex cross stitch that is left long as frayed fringe. Featuring a roll neck and long voluminous balloon sleeves with an invisible zipper on center back.",

"images": "https://cdn.yoox.biz/items/34/34767069qs_18_g_f.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_r.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_d.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_e.jpg|https://cdn.yoox.biz/items/34/34767069qs_18_g_a.jpg",

"name": "Long-Sleeved Knit Dress With Roll Neck",

"price": 2669,

"sku": "493373Q1WHB1056",

"url": "http://www.alexandermcqueen.com/us/alexandermcqueen/long-dress_cod34767069qs.html",

"variations": "BLACK/RED"

}

}]

Михаил Сисин

Бесплатный парсер товаров онлайн магазина Alexander McQueen

Добавить комментарий Отменить ответ

Свежие записи

Архивы

Рубрики