How to project an alias using a wildcard?

Once I do a join A by id, B by id, I get an alias with fields A::f..., B::f... Is there a way to project it on only the A fields?

C = join A by id, B by id; D = filter C by B::n < 1000; E = foreach D generate A::*;

I get

Unexpected character '*'

What I want is E with the schema identical to A (i.e., describe E and describe A should print the exact same things).

How do I do that?

Answer1:

You can use a project-range expression to get part of the way there.

Unfortunately, there is no way to systematically strip the A:: prefix. If you know the name of the last field of A (suppose it's last), you can do this:

E = foreach D generate .. A::last;

If you wanted just the fields from B you would do

E = foreach D generate B::first ..;

If you really need to apply a specific schema, perhaps you could just define a macro that applies this schema whenever you need it, so you can overwrite any of the changes that come from grouping, joining, etc.

Answer2:

There is no way to have a common alias name after joining. but you can generate specific columns from the join results. For Example,

A = load 'data1' as (id,name,addr); B = load 'data2' as (id,name2,addr2); C = join A by id,B by id; //Now C has id,name,addr,id,name2,addr2 D = Foreach C generate($0,$1,$2);

Now the relation D has the 'A' relation columns such as id,name,addr only.

人吐槽 人点赞

Recommend

Comment

用户名: 密码:
验证码: 匿名发表

你可以使用这些语言

查看评论:How to project an alias using a wildcard?